[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:08 dist-test-slave-rmhg krb5kdc[7336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:08 dist-test-slave-rmhg krb5kdc[7336](info): set up 2 sockets
Aug 11 12:15:08 dist-test-slave-rmhg krb5kdc[7336](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 12:15:09.871865 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.008s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:10 dist-test-slave-rmhg krb5kdc[7336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:10.088140 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:36633
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:36633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:10.490871 7354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:10.491325 7354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:10.491494 7354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:10.499619 7354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:10.499770 7354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:10.499832 7354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:10.499915 7354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:10.499992 7354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:10.511498 7354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:36633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:36633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:10.513711 7354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:10.515877 7354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:10.536809 7362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:10.537561 7360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:10.537985 7370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:10.539039 7354 server_base.cc:1034] running on GCE node
I20260811 12:15:10.540419 7354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:10.543061 7354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:10.544539 7354 hybrid_clock.cc:648] HybridClock initialized: now 1786450510544422 us; error 113 us; skew 500 ppm
I20260811 12:15:10.545105 7354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:10 dist-test-slave-rmhg krb5kdc[7336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:10.552551 7354 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:10.555626 7354 webserver.cc:460] Webserver started at http://127.7.39.62:36387/ using document root <none> and password file <none>
I20260811 12:15:10.556939 7354 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:10.557080 7354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:10.557677 7354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:10.562320 7354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "e81b1e634e9b4625aa81ce4984c9e9f7"
format_stamp: "Formatted at 2026-08-11 12:15:10 on dist-test-slave-rmhg"
server_key: "a9d868f5e8b0cb00a165bb73fa4e31c9"
server_key_iv: "961fdc8ac52df6db063fb91cf202cea0"
server_key_version: "encryptionkey@0"
I20260811 12:15:10.563382 7354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "e81b1e634e9b4625aa81ce4984c9e9f7"
format_stamp: "Formatted at 2026-08-11 12:15:10 on dist-test-slave-rmhg"
server_key: "a9d868f5e8b0cb00a165bb73fa4e31c9"
server_key_iv: "961fdc8ac52df6db063fb91cf202cea0"
server_key_version: "encryptionkey@0"
I20260811 12:15:10.572705 7354 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260811 12:15:10.579401 7389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:10.581705 7354 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260811 12:15:10.582005 7354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "e81b1e634e9b4625aa81ce4984c9e9f7"
format_stamp: "Formatted at 2026-08-11 12:15:10 on dist-test-slave-rmhg"
server_key: "a9d868f5e8b0cb00a165bb73fa4e31c9"
server_key_iv: "961fdc8ac52df6db063fb91cf202cea0"
server_key_version: "encryptionkey@0"
I20260811 12:15:10.582252 7354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:10.598851 7354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:10.604187 7354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:10.604555 7354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:10.628189 7354 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:36633
I20260811 12:15:10.628208 7469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:36633 every 8 connection(s)
I20260811 12:15:10.630935 7354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:10.632265 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 7354
I20260811 12:15:10.632720 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:10.635082 7324 external_mini_cluster.cc:1428] Setting key 83f242dfc29ae12a8b4f9159d0641be3
I20260811 12:15:10.638873 7470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:10.656620 7470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7: Bootstrap starting.
I20260811 12:15:10.662967 7470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:10.664799 7470 log.cc:824] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:15:10 dist-test-slave-rmhg krb5kdc[7336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:10.669183 7470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7: No bootstrap required, opened a new log
I20260811 12:15:10.679584 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:10.641695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.7.39.62:36633)
0811 12:15:10.642969 (+ 1274us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:10.642991 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:10.644384 (+ 1393us) server_negotiation.cc:407] Connection header received
0811 12:15:10.654193 (+ 9809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:10.654219 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:10.654899 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:10.655384 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:10.658940 (+ 3556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:10.660442 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:10.662349 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:10.663142 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:10.663411 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:10.669407 (+ 5996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:10.669500 (+ 93us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:10.669538 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:10.669582 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:10.673015 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:10.674067 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:10.674078 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:10.674085 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:10.674330 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:10.674845 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:10.674849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:10.674851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:10.675645 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:10.676134 (+ 489us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:10.677232 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0811 12:15:10.677728 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":442,"threads_started":1}
I20260811 12:15:10.679456 7470 raft_consensus.cc:348] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81b1e634e9b4625aa81ce4984c9e9f7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 36633 } }
I20260811 12:15:10.680086 7470 raft_consensus.cc:374] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:10.680287 7470 raft_consensus.cc:729] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e81b1e634e9b4625aa81ce4984c9e9f7, State: Initialized, Role: FOLLOWER
I20260811 12:15:10.681689 7470 consensus_queue.cc:260] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81b1e634e9b4625aa81ce4984c9e9f7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 36633 } }
I20260811 12:15:10.682090 7470 raft_consensus.cc:388] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:10.682477 7470 raft_consensus.cc:482] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:10.682718 7470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:10.685621 7470 raft_consensus.cc:504] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81b1e634e9b4625aa81ce4984c9e9f7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 36633 } }
I20260811 12:15:10.686674 7470 leader_election.cc:302] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e81b1e634e9b4625aa81ce4984c9e9f7; no voters:
I20260811 12:15:10.687525 7470 leader_election.cc:288] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:10.688000 7475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:10.689077 7475 raft_consensus.cc:686] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [term 1 LEADER]: Becoming Leader. State: Replica: e81b1e634e9b4625aa81ce4984c9e9f7, State: Running, Role: LEADER
I20260811 12:15:10.689941 7475 consensus_queue.cc:237] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81b1e634e9b4625aa81ce4984c9e9f7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 36633 } }
I20260811 12:15:10.690966 7470 sys_catalog.cc:564] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:10.695355 7481 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e81b1e634e9b4625aa81ce4984c9e9f7. Latest consensus state: current_term: 1 leader_uuid: "e81b1e634e9b4625aa81ce4984c9e9f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81b1e634e9b4625aa81ce4984c9e9f7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 36633 } } }
I20260811 12:15:10.695853 7481 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:10.695379 7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e81b1e634e9b4625aa81ce4984c9e9f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81b1e634e9b4625aa81ce4984c9e9f7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 36633 } } }
I20260811 12:15:10.696758 7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:10.699630 7489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:10.714100 7489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:10.730275 7489 catalog_manager.cc:1347] Generated new cluster ID: b01e86517bf74e9baf513757e0f79008
I20260811 12:15:10.730418 7489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:10.751196 7489 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:10.752846 7489 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:10.760108 7489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e81b1e634e9b4625aa81ce4984c9e9f7: Generated new TSK 0
I20260811 12:15:10.761425 7489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:15:10.813768 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:36633
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:11.307760 7517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:11.308318 7517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:11.308573 7517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:11.318560 7517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:11.318823 7517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:11.319082 7517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:11.332530 7517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:36633
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:11.334122 7517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:11.336028 7517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:11.353386 7529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:11.353639 7525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:11.354208 7524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:11.355176 7517 server_base.cc:1034] running on GCE node
I20260811 12:15:11.355896 7517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:11.357239 7517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:11.358527 7517 hybrid_clock.cc:648] HybridClock initialized: now 1786450511358417 us; error 50 us; skew 500 ppm
I20260811 12:15:11.358870 7517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:11 dist-test-slave-rmhg krb5kdc[7336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450511, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:11.363715 7517 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:15:11.365796 7517 webserver.cc:460] Webserver started at http://127.7.39.1:39793/ using document root <none> and password file <none>
I20260811 12:15:11.366503 7517 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:11.366626 7517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:11.366989 7517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:11.371030 7517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "9a0a932e21e04843a6c905671e0bec61"
format_stamp: "Formatted at 2026-08-11 12:15:11 on dist-test-slave-rmhg"
server_key: "6a209d91578e881402d98ba5ec5ea469"
server_key_iv: "bd3c41fa109b4b4861f30f1a0523ff42"
server_key_version: "encryptionkey@0"
I20260811 12:15:11.372176 7517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "9a0a932e21e04843a6c905671e0bec61"
format_stamp: "Formatted at 2026-08-11 12:15:11 on dist-test-slave-rmhg"
server_key: "6a209d91578e881402d98ba5ec5ea469"
server_key_iv: "bd3c41fa109b4b4861f30f1a0523ff42"
server_key_version: "encryptionkey@0"
I20260811 12:15:11.379621 7517 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260811 12:15:11.384477 7545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:11.386119 7517 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 12:15:11.386353 7517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "9a0a932e21e04843a6c905671e0bec61"
format_stamp: "Formatted at 2026-08-11 12:15:11 on dist-test-slave-rmhg"
server_key: "6a209d91578e881402d98ba5ec5ea469"
server_key_iv: "bd3c41fa109b4b4861f30f1a0523ff42"
server_key_version: "encryptionkey@0"
I20260811 12:15:11.386740 7517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:11.405892 7517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:11.412227 7517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:11.412765 7517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:11.414563 7517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:11.417104 7517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:11.417304 7517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 12:15:11.417589 7517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:11.417722 7517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:11.498601 7517 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:34801
I20260811 12:15:11.498700 7736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:34801 every 8 connection(s)
I20260811 12:15:11.500584 7517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:11.509852 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 7517
I20260811 12:15:11.510195 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:11.511019 7324 external_mini_cluster.cc:1428] Setting key 400ab7bb7da4a23e28f3a18fc6748e43
Aug 11 12:15:11 dist-test-slave-rmhg krb5kdc[7336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450511, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:11.530606 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:11.504983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:48307 (local address 127.7.39.62:36633)
0811 12:15:11.505708 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:11.505713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:11.507225 (+ 1512us) server_negotiation.cc:407] Connection header received
0811 12:15:11.510023 (+ 2798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:11.510029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:11.510181 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:11.510289 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:11.512815 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:11.513825 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:11.515949 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:11.516464 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:11.516589 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:11.522059 (+ 5470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:11.522065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:11.522068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:11.522107 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:11.525842 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:11.527202 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:11.527208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:11.527210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:11.527475 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:11.528022 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:11.528025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:11.528027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:11.528398 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:11.528521 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:11.529624 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0811 12:15:11.530079 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":441,"threads_started":1}
I20260811 12:15:11.532423 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:11.505885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:36633 (local address 127.7.39.1:48307)
0811 12:15:11.507041 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:11.507096 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:11.509714 (+ 2618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:11.510667 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:11.510677 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:11.511559 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:11.512567 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:11.512593 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:11.513976 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:11.513980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:11.515783 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:11.515798 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:11.516817 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:11.516826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:11.517193 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:11.518248 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:11.518278 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:11.521756 (+ 3478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:11.526053 (+ 4297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:11.526064 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:11.526086 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:11.527001 (+ 915us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:11.527618 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:11.527623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:11.527626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:11.527917 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:11.528555 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:11.528564 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:11.529044 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 12:15:11.529494 (+ 450us) client_negotiation.cc:240] Negotiation successful
0811 12:15:11.530372 (+ 878us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":542,"threads_started":1}
I20260811 12:15:11.534379 7737 heartbeater.cc:347] Connected to a master server at 127.7.39.62:36633
I20260811 12:15:11.535015 7737 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
I20260811 12:15:11.536427 7737 heartbeater.cc:511] Master 127.7.39.62:36633 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
I20260811 12:15:11.541250 7415 ts_manager.cc:194] Registered new tserver with Master: 9a0a932e21e04843a6c905671e0bec61 (127.7.39.1:34801)
I20260811 12:15:11.543835 7415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:48307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:15:11.564585 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:36633
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:12.171940 7750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:12.172382 7750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:12.172614 7750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:12.182770 7750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:12.183053 7750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:12.183321 7750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:12.196796 7750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:36633
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:12.198305 7750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:12.200802 7750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:12.219880 7760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:12.220180 7761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:12.220202 7766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:12.222126 7750 server_base.cc:1034] running on GCE node
I20260811 12:15:12.222895 7750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:12.224110 7750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:12.225363 7750 hybrid_clock.cc:648] HybridClock initialized: now 1786450512225259 us; error 54 us; skew 500 ppm
I20260811 12:15:12.225732 7750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:12 dist-test-slave-rmhg krb5kdc[7336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450512, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:12.229892 7750 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:15:12.232162 7750 webserver.cc:460] Webserver started at http://127.7.39.2:44967/ using document root <none> and password file <none>
I20260811 12:15:12.233112 7750 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:12.233381 7750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:12.233829 7750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:12.237661 7750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "9172c9c716d74759bbcf7dad7f78ac2f"
format_stamp: "Formatted at 2026-08-11 12:15:12 on dist-test-slave-rmhg"
server_key: "47fe7ab59dc86cd3d39770e0713a857a"
server_key_iv: "8f264ccb7e653b7290aa2235b43a53ae"
server_key_version: "encryptionkey@0"
I20260811 12:15:12.238456 7750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "9172c9c716d74759bbcf7dad7f78ac2f"
format_stamp: "Formatted at 2026-08-11 12:15:12 on dist-test-slave-rmhg"
server_key: "47fe7ab59dc86cd3d39770e0713a857a"
server_key_iv: "8f264ccb7e653b7290aa2235b43a53ae"
server_key_version: "encryptionkey@0"
I20260811 12:15:12.245602 7750 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.002s
I20260811 12:15:12.250799 7777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:12.252295 7750 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260811 12:15:12.252471 7750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "9172c9c716d74759bbcf7dad7f78ac2f"
format_stamp: "Formatted at 2026-08-11 12:15:12 on dist-test-slave-rmhg"
server_key: "47fe7ab59dc86cd3d39770e0713a857a"
server_key_iv: "8f264ccb7e653b7290aa2235b43a53ae"
server_key_version: "encryptionkey@0"
I20260811 12:15:12.252772 7750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:12.283675 7750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:12.290210 7750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:12.290720 7750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:12.292502 7750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:12.295619 7750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:12.295780 7750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:12.296087 7750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:12.296166 7750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:12.375972 7750 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:32985
I20260811 12:15:12.376153 7949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:32985 every 8 connection(s)
I20260811 12:15:12.377914 7750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:12.382365 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 7750
I20260811 12:15:12.382679 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:12.383397 7324 external_mini_cluster.cc:1428] Setting key 6dd4509fb7e246f9f9bd5aca5b10af50
Aug 11 12:15:12 dist-test-slave-rmhg krb5kdc[7336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450512, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
I20260811 12:15:12.411708 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:12.384391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:40813 (local address 127.7.39.62:36633)
0811 12:15:12.385359 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:12.385365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:12.389245 (+ 3880us) server_negotiation.cc:407] Connection header received
0811 12:15:12.389794 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:12.389800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:12.389958 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:12.390098 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:12.392515 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:12.393700 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:12.396933 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:12.397609 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:12.397727 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:12.403003 (+ 5276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:12.403012 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:12.403016 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:12.403068 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:12.407914 (+ 4846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:12.408868 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:12.408874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:12.408877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:12.409089 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:12.409673 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:12.409677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:12.409680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:12.410106 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:12.410242 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:12.411076 (+ 834us) server_negotiation.cc:299] Negotiation successful
0811 12:15:12.411331 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":853,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
I20260811 12:15:12.413213 7954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:12.383247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:36633 (local address 127.7.39.2:40813)
0811 12:15:12.386393 (+ 3146us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:12.386471 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:12.389461 (+ 2990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:12.390346 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:12.390356 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:12.391244 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:12.392300 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:12.392326 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:12.393906 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:12.393910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:12.396745 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:12.396762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:12.397870 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:12.397877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:12.398198 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:12.399138 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:12.399173 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:12.402670 (+ 3497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:12.408113 (+ 5443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:12.408123 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:12.408140 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:12.408718 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:12.409235 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:12.409238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:12.409241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:12.409515 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:12.410256 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:12.410264 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:12.410638 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 12:15:12.410930 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 12:15:12.411635 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2585,"thread_start_us":2775,"threads_started":1}
I20260811 12:15:12.414969 7951 heartbeater.cc:347] Connected to a master server at 127.7.39.62:36633
I20260811 12:15:12.415352 7951 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:12.416107 7951 heartbeater.cc:511] Master 127.7.39.62:36633 requested a full tablet report, sending...
I20260811 12:15:12.417706 7415 ts_manager.cc:194] Registered new tserver with Master: 9172c9c716d74759bbcf7dad7f78ac2f (127.7.39.2:32985)
I20260811 12:15:12.418746 7415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:40813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:15:12.440290 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:36633
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:15:12.549177 7737 heartbeater.cc:503] Master 127.7.39.62:36633 was elected leader, sending a full tablet report...
W20260811 12:15:13.042076 7960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:13.042506 7960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:13.042707 7960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:13.052273 7960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:13.052439 7960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:13.052654 7960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:13.066277 7960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:36633
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:13.067665 7960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:13.069394 7960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:13.088455 7975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:13.088877 7977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:13.089447 7982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:13.090119 7960 server_base.cc:1034] running on GCE node
I20260811 12:15:13.090819 7960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:13.092131 7960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:13.093473 7960 hybrid_clock.cc:648] HybridClock initialized: now 1786450513093376 us; error 44 us; skew 500 ppm
I20260811 12:15:13.093906 7960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:13 dist-test-slave-rmhg krb5kdc[7336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450513, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:13.098964 7960 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:15:13.101073 7960 webserver.cc:460] Webserver started at http://127.7.39.3:39587/ using document root <none> and password file <none>
I20260811 12:15:13.102099 7960 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:13.102275 7960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:13.102610 7960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:13.106372 7960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "874e4ccf5a90475ea13e48f9da0564d7"
format_stamp: "Formatted at 2026-08-11 12:15:13 on dist-test-slave-rmhg"
server_key: "ec3c44e69eb843dae71968c31ee6b1ba"
server_key_iv: "c07ba21cf66a29f777de50399637439b"
server_key_version: "encryptionkey@0"
I20260811 12:15:13.107429 7960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "874e4ccf5a90475ea13e48f9da0564d7"
format_stamp: "Formatted at 2026-08-11 12:15:13 on dist-test-slave-rmhg"
server_key: "ec3c44e69eb843dae71968c31ee6b1ba"
server_key_iv: "c07ba21cf66a29f777de50399637439b"
server_key_version: "encryptionkey@0"
I20260811 12:15:13.115298 7960 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260811 12:15:13.120659 7992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:13.122346 7960 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 12:15:13.122653 7960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "874e4ccf5a90475ea13e48f9da0564d7"
format_stamp: "Formatted at 2026-08-11 12:15:13 on dist-test-slave-rmhg"
server_key: "ec3c44e69eb843dae71968c31ee6b1ba"
server_key_iv: "c07ba21cf66a29f777de50399637439b"
server_key_version: "encryptionkey@0"
I20260811 12:15:13.123121 7960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:13.160106 7960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:13.166070 7960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:13.166386 7960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:13.168001 7960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:13.170382 7960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:13.170529 7960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:13.170769 7960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:13.170914 7960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:13.249656 7960 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:40111
I20260811 12:15:13.249717 8176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:40111 every 8 connection(s)
I20260811 12:15:13.251116 7960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:13.258718 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 7960
I20260811 12:15:13.258915 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:13.259418 7324 external_mini_cluster.cc:1428] Setting key c6166eccb49269f0cd3342e934cc9b90
Aug 11 12:15:13 dist-test-slave-rmhg krb5kdc[7336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450513, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:13.280017 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:13.255010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:52719 (local address 127.7.39.62:36633)
0811 12:15:13.255700 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:13.255705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:13.257585 (+ 1880us) server_negotiation.cc:407] Connection header received
0811 12:15:13.260690 (+ 3105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:13.260696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:13.260898 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:13.261033 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:13.263287 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.264187 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.266244 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.266857 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.266960 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:13.271338 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:13.271344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:13.271348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:13.271384 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:13.275367 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.276537 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.276542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.276545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.276782 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.277550 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.277556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.277559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.278045 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:13.278208 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:13.279066 (+ 858us) server_negotiation.cc:299] Negotiation successful
0811 12:15:13.279361 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":454,"threads_started":1}
I20260811 12:15:13.281093 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:13.255884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:36633 (local address 127.7.39.3:52719)
0811 12:15:13.257416 (+ 1532us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:13.257461 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:13.260343 (+ 2882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:13.261389 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:13.261402 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:13.262131 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:13.263086 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:13.263117 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.264387 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.264392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:13.266103 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:13.266117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.267087 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.267094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:13.267406 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:13.268116 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:13.268141 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:13.271034 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:13.275586 (+ 4552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:13.275594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:13.275612 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:13.276390 (+ 778us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:13.276906 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:13.276911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:13.276914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:13.277375 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:13.278217 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:13.278227 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:13.278643 (+ 416us) client_negotiation.cc:769] Sending connection context
0811 12:15:13.278943 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 12:15:13.279610 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1163,"thread_start_us":799,"threads_started":1}
I20260811 12:15:13.282660 8177 heartbeater.cc:347] Connected to a master server at 127.7.39.62:36633
I20260811 12:15:13.283049 8177 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:13.284149 8177 heartbeater.cc:511] Master 127.7.39.62:36633 requested a full tablet report, sending...
I20260811 12:15:13.285966 7415 ts_manager.cc:194] Registered new tserver with Master: 874e4ccf5a90475ea13e48f9da0564d7 (127.7.39.3:40111)
I20260811 12:15:13.287173 7415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:52719
I20260811 12:15:13.290364 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:15:13 dist-test-slave-rmhg krb5kdc[7336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450513, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:15:13 dist-test-slave-rmhg krb5kdc[7336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450513, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:13.363420 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:13.342004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50494 (local address 127.7.39.1:34801)
0811 12:15:13.342965 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:13.342975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:13.343174 (+ 199us) server_negotiation.cc:407] Connection header received
0811 12:15:13.343381 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:13.343388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:13.343632 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:13.343933 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:13.346305 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.347336 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.349090 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.349761 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.349886 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:13.356033 (+ 6147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:13.356051 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:13.356070 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:13.356111 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:13.360131 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.360930 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.360938 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.360941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.361076 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.361519 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.361523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.361525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.361921 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:13.362437 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:13.362689 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 12:15:13.363017 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":418,"threads_started":1}
W20260811 12:15:13.365193 7594 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50494
Aug 11 12:15:13 dist-test-slave-rmhg krb5kdc[7336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:15:13 dist-test-slave-rmhg krb5kdc[7336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:13.422235 7951 heartbeater.cc:503] Master 127.7.39.62:36633 was elected leader, sending a full tablet report...
I20260811 12:15:13.428086 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:13.410545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.7.39.1:34801)
0811 12:15:13.410792 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:13.410799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:13.411260 (+ 461us) server_negotiation.cc:407] Connection header received
0811 12:15:13.411449 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:13.411453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:13.411521 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:13.411606 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:13.413852 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.414561 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.415935 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.416408 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.416528 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:13.422223 (+ 5695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:13.422229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:13.422233 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:13.422274 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:13.425236 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.426050 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.426056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.426058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.426206 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.426566 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.426570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.426572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.426834 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:13.427062 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:13.427476 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 12:15:13.427752 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 12:15:13.430365 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 7517
I20260811 12:15:13.447566 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:13.431561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.7.39.1:34801)
0811 12:15:13.431824 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:13.431829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:13.431971 (+ 142us) server_negotiation.cc:407] Connection header received
0811 12:15:13.432098 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:13.432102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:13.432164 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:13.432241 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:13.434100 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.435030 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.436828 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.437476 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.437578 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:13.440643 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:13.440650 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:13.440653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:13.440691 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:13.444468 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.445394 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.445398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.445401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.445627 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.445970 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.445973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.445976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.446279 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:13.446481 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:13.446791 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 12:15:13.446968 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:15:13.448462 7729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:13.541596 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 7517
I20260811 12:15:13.567797 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 7750
Aug 11 12:15:13 dist-test-slave-rmhg krb5kdc[7336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:15:13.584738 8207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:13.568810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.7.39.2:32985)
0811 12:15:13.569619 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:13.569627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:13.569653 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:15:13.569713 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:13.569720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:13.569936 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:13.570086 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:13.571268 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.572332 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.573752 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.574361 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.574475 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:13.578834 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:13.578848 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:13.578866 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:13.578906 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:13.581647 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.582326 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.582334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.582336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.582473 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.582909 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.582914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.582916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.583401 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:13.583718 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:13.584066 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 12:15:13.584346 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":497,"threads_started":1}
I20260811 12:15:13.586254 7944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:13.669270 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 7750
I20260811 12:15:13.692302 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 7960
Aug 11 12:15:13 dist-test-slave-rmhg krb5kdc[7336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:15:13.709419 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:13.693493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.7.39.3:40111)
0811 12:15:13.694084 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:13.694092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:13.694124 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:15:13.694188 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:13.694194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:13.694413 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:13.694538 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:13.695747 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.696733 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.698183 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:13.698815 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:13.698927 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:13.703343 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:13.703362 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:13.703379 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:13.703429 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:13.706350 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.707117 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.707133 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.707135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.707278 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:13.707675 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:13.707680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:13.707682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:13.708071 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:13.708387 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:13.708723 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 12:15:13.709000 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":307,"threads_started":1}
I20260811 12:15:13.710769 8166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:13.795966 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 7960
I20260811 12:15:13.832248 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 7354
I20260811 12:15:13.833712 7464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:13.941665 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 7354
2026-08-11T12:15:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5327 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:14 dist-test-slave-rmhg krb5kdc[8216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:14 dist-test-slave-rmhg krb5kdc[8216](info): set up 2 sockets
Aug 11 12:15:14 dist-test-slave-rmhg krb5kdc[8216](info): commencing operation
krb5kdc: starting...
W20260811 12:15:15.172328 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:15 dist-test-slave-rmhg krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:15.382668 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:43619
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:40919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:43619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:15.720603 8237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:15.720986 8237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:15.721138 8237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:15.728566 8237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:15.728708 8237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:15.728801 8237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:15.728884 8237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:15.728950 8237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:15.739274 8237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:40919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:43619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:43619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:15.740718 8237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:15.742208 8237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:15.754333 8245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:15.754295 8248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:15.754379 8246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:15.755271 8237 server_base.cc:1034] running on GCE node
I20260811 12:15:15.756085 8237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:15.757615 8237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:15.758829 8237 hybrid_clock.cc:648] HybridClock initialized: now 1786450515758776 us; error 48 us; skew 500 ppm
I20260811 12:15:15.759191 8237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:15 dist-test-slave-rmhg krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450515, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:15.770223 8237 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:15.771947 8237 webserver.cc:460] Webserver started at http://127.7.39.62:33699/ using document root <none> and password file <none>
I20260811 12:15:15.772691 8237 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:15.772801 8237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:15.773077 8237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:15.776342 8237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "4025570d03944973a761610442b64ddb"
format_stamp: "Formatted at 2026-08-11 12:15:15 on dist-test-slave-rmhg"
server_key: "a3ee9ce457ae4ab6d7f401021a6e4326"
server_key_iv: "bce5debbba223a2d5c65dcf9a55ed78b"
server_key_version: "encryptionkey@0"
I20260811 12:15:15.777133 8237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "4025570d03944973a761610442b64ddb"
format_stamp: "Formatted at 2026-08-11 12:15:15 on dist-test-slave-rmhg"
server_key: "a3ee9ce457ae4ab6d7f401021a6e4326"
server_key_iv: "bce5debbba223a2d5c65dcf9a55ed78b"
server_key_version: "encryptionkey@0"
I20260811 12:15:15.782742 8237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:15:15.786646 8258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:15.788170 8237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:15.788341 8237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "4025570d03944973a761610442b64ddb"
format_stamp: "Formatted at 2026-08-11 12:15:15 on dist-test-slave-rmhg"
server_key: "a3ee9ce457ae4ab6d7f401021a6e4326"
server_key_iv: "bce5debbba223a2d5c65dcf9a55ed78b"
server_key_version: "encryptionkey@0"
I20260811 12:15:15.788542 8237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:15.802913 8237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:15.807421 8237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:15.807717 8237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:15.825663 8237 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:43619
I20260811 12:15:15.825704 8334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:43619 every 8 connection(s)
I20260811 12:15:15.827111 8237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:15.831085 8335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:15.833370 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8237
I20260811 12:15:15.833616 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:15.834431 7324 external_mini_cluster.cc:1428] Setting key 89c4b6ce7d84609cfdde2b283044690c
I20260811 12:15:15.843487 8335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb: Bootstrap starting.
I20260811 12:15:15.846958 8335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:15.847977 8335 log.cc:824] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:15:15 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:15.851111 8335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb: No bootstrap required, opened a new log
I20260811 12:15:15.857376 8340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:15.836885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52422 (local address 127.7.39.62:43619)
0811 12:15:15.837777 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:15.837790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:15.837834 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:15:15.839548 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:15.839574 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:15.840004 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:15.840393 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:15.842002 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:15.843345 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:15.844756 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:15.845552 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:15.845750 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:15.849966 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:15.849974 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:15.849990 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:15.850027 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:15.853067 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:15.853813 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:15.853820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:15.853827 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:15.854010 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:15.854382 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:15.854386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:15.854388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:15.854963 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:15.855289 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:15.855786 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 12:15:15.856237 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":405,"threads_started":1}
I20260811 12:15:15.859535 8335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4025570d03944973a761610442b64ddb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43619 } }
I20260811 12:15:15.859915 8335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:15.860016 8335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4025570d03944973a761610442b64ddb, State: Initialized, Role: FOLLOWER
I20260811 12:15:15.860656 8335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4025570d03944973a761610442b64ddb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43619 } }
I20260811 12:15:15.860904 8335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:15.861043 8335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:15.861232 8335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:15.863410 8335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4025570d03944973a761610442b64ddb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43619 } }
I20260811 12:15:15.863966 8335 leader_election.cc:302] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4025570d03944973a761610442b64ddb; no voters:
I20260811 12:15:15.864529 8335 leader_election.cc:288] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:15.864658 8343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:15.865317 8343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [term 1 LEADER]: Becoming Leader. State: Replica: 4025570d03944973a761610442b64ddb, State: Running, Role: LEADER
I20260811 12:15:15.865849 8343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4025570d03944973a761610442b64ddb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43619 } }
I20260811 12:15:15.866556 8335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:15.868592 8347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4025570d03944973a761610442b64ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4025570d03944973a761610442b64ddb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43619 } } }
I20260811 12:15:15.868973 8347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:15.870031 8348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4025570d03944973a761610442b64ddb. Latest consensus state: current_term: 1 leader_uuid: "4025570d03944973a761610442b64ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4025570d03944973a761610442b64ddb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43619 } } }
I20260811 12:15:15.870419 8348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:15.871441 8356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:15.876686 8356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:15.885550 8356 catalog_manager.cc:1347] Generated new cluster ID: ae6d4ae19e5448ea9aedb9c8946ac360
I20260811 12:15:15.885671 8356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:15.901428 8356 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:15.903339 8356 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:15.918428 8356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4025570d03944973a761610442b64ddb: Generated new TSK 0
I20260811 12:15:15.919601 8356 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:15:15.978281 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43619
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:40919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:16.317765 8378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:16.318110 8378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:16.318259 8378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:16.325824 8378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:16.325978 8378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:16.326138 8378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:16.336383 8378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:40919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43619
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:16.337896 8378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:16.339332 8378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:16.352895 8389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:16.353003 8386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:16.352977 8392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:16.354054 8378 server_base.cc:1034] running on GCE node
I20260811 12:15:16.354764 8378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:16.355875 8378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:16.357137 8378 hybrid_clock.cc:648] HybridClock initialized: now 1786450516357054 us; error 78 us; skew 500 ppm
I20260811 12:15:16.357546 8378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:16 dist-test-slave-rmhg krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450516, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:16.360932 8378 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:15:16.362635 8378 webserver.cc:460] Webserver started at http://127.7.39.1:42419/ using document root <none> and password file <none>
I20260811 12:15:16.363368 8378 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:16.363497 8378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:16.363812 8378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:16.367070 8378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "f33b631f0ab2497b960e8a7fb828075a"
format_stamp: "Formatted at 2026-08-11 12:15:16 on dist-test-slave-rmhg"
server_key: "43d9b08e9e2148743f788e4e73d4806b"
server_key_iv: "1fabc59d1db4dcaca9828d0b49173147"
server_key_version: "encryptionkey@0"
I20260811 12:15:16.367878 8378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "f33b631f0ab2497b960e8a7fb828075a"
format_stamp: "Formatted at 2026-08-11 12:15:16 on dist-test-slave-rmhg"
server_key: "43d9b08e9e2148743f788e4e73d4806b"
server_key_iv: "1fabc59d1db4dcaca9828d0b49173147"
server_key_version: "encryptionkey@0"
I20260811 12:15:16.373986 8378 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 12:15:16.378578 8402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:16.380322 8378 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:15:16.380509 8378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "f33b631f0ab2497b960e8a7fb828075a"
format_stamp: "Formatted at 2026-08-11 12:15:16 on dist-test-slave-rmhg"
server_key: "43d9b08e9e2148743f788e4e73d4806b"
server_key_iv: "1fabc59d1db4dcaca9828d0b49173147"
server_key_version: "encryptionkey@0"
I20260811 12:15:16.380771 8378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:16.397209 8378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:16.401659 8378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:16.401933 8378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:16.402928 8378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:16.404390 8378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:16.404491 8378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:16.404595 8378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:16.404670 8378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:16.436565 8378 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:39813
I20260811 12:15:16.436617 8591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:39813 every 8 connection(s)
I20260811 12:15:16.437949 8378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:16.440362 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8378
I20260811 12:15:16.440590 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:16.441361 7324 external_mini_cluster.cc:1428] Setting key 69f39aa4b40b625e1552a46459feaa41
Aug 11 12:15:16 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450516, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:16.460119 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:16.440622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:37915 (local address 127.7.39.62:43619)
0811 12:15:16.441182 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:16.441188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:16.442317 (+ 1129us) server_negotiation.cc:407] Connection header received
0811 12:15:16.444361 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:16.444367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:16.444472 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:16.444592 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:16.446259 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:16.447261 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:16.448882 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:16.449481 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:16.449610 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:16.454025 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:16.454031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:16.454034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:16.454072 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:16.456709 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:16.457621 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:16.457625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:16.457628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:16.457763 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:16.458226 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:16.458230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:16.458232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:16.458530 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:16.458672 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:16.459477 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 12:15:16.459733 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":248,"threads_started":1}
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
I20260811 12:15:16.461210 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:16.441265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43619 (local address 127.7.39.1:37915)
0811 12:15:16.442156 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:16.442206 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:16.444060 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:16.444813 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:16.444822 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:16.445443 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:16.446062 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:16.446093 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:16.447437 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:16.447441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:16.448726 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:16.448739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:16.449737 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:16.449746 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:16.450075 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:16.450866 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:16.450905 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:16.453721 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:16.456892 (+ 3171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:16.456904 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:16.456920 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:16.457474 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:16.457880 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:16.457884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:16.457887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:16.458099 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:16.458667 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:16.458676 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:16.459072 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 12:15:16.459333 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 12:15:16.459936 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":351,"threads_started":1}
I20260811 12:15:16.462890 8594 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
I20260811 12:15:16.463253 8594 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:16.464026 8594 heartbeater.cc:511] Master 127.7.39.62:43619 requested a full tablet report, sending...
I20260811 12:15:16.466800 8280 ts_manager.cc:194] Registered new tserver with Master: f33b631f0ab2497b960e8a7fb828075a (127.7.39.1:39813)
I20260811 12:15:16.468780 8280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:37915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:15:16.489068 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43619
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:40919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:16.828781 8605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:16.829075 8605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:16.829222 8605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:16.837128 8605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:16.837277 8605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:16.837455 8605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:16.847728 8605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:40919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43619
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:16.849138 8605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:16.850657 8605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:16.864379 8616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:16.864569 8611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:16.864359 8612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:16.865223 8605 server_base.cc:1034] running on GCE node
I20260811 12:15:16.865907 8605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:16.866978 8605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:16.868194 8605 hybrid_clock.cc:648] HybridClock initialized: now 1786450516868127 us; error 55 us; skew 500 ppm
I20260811 12:15:16.868528 8605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:16 dist-test-slave-rmhg krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450516, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:16.871968 8605 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:15:16.873683 8605 webserver.cc:460] Webserver started at http://127.7.39.2:33901/ using document root <none> and password file <none>
I20260811 12:15:16.874402 8605 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:16.874534 8605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:16.874838 8605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:16.878041 8605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "c30e45720dfe418a9c779e25631f17ba"
format_stamp: "Formatted at 2026-08-11 12:15:16 on dist-test-slave-rmhg"
server_key: "edadafe48da7eef9f6cd22431b7baec8"
server_key_iv: "e887f4df1243c552a63ae6da82b80f46"
server_key_version: "encryptionkey@0"
I20260811 12:15:16.878861 8605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "c30e45720dfe418a9c779e25631f17ba"
format_stamp: "Formatted at 2026-08-11 12:15:16 on dist-test-slave-rmhg"
server_key: "edadafe48da7eef9f6cd22431b7baec8"
server_key_iv: "e887f4df1243c552a63ae6da82b80f46"
server_key_version: "encryptionkey@0"
I20260811 12:15:16.884474 8605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:15:16.888520 8627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:16.889976 8605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:16.890142 8605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "c30e45720dfe418a9c779e25631f17ba"
format_stamp: "Formatted at 2026-08-11 12:15:16 on dist-test-slave-rmhg"
server_key: "edadafe48da7eef9f6cd22431b7baec8"
server_key_iv: "e887f4df1243c552a63ae6da82b80f46"
server_key_version: "encryptionkey@0"
I20260811 12:15:16.890333 8605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:16.904844 8605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:16.909307 8605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:16.909606 8605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:16.910609 8605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:16.912019 8605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:16.912110 8605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:16.912238 8605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:16.912298 8605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:16.942714 8605 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:37313
I20260811 12:15:16.942760 8816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:37313 every 8 connection(s)
I20260811 12:15:16.943990 8605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:16.951815 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8605
I20260811 12:15:16.952030 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:16.952579 7324 external_mini_cluster.cc:1428] Setting key c78785cea78dc4d3dce70869315184e2
Aug 11 12:15:16 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450516, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:16.966007 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:16.946621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:47113 (local address 127.7.39.62:43619)
0811 12:15:16.946827 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:16.946833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:16.948202 (+ 1369us) server_negotiation.cc:407] Connection header received
0811 12:15:16.950055 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:16.950060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:16.950153 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:16.950244 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:16.951801 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:16.952883 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:16.955289 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:16.955919 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:16.956051 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:16.959887 (+ 3836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:16.959894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:16.959898 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:16.959938 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:16.962706 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:16.963630 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:16.963636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:16.963640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:16.963769 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:16.964255 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:16.964258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:16.964261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:16.964545 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:16.964660 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:16.965563 (+ 903us) server_negotiation.cc:299] Negotiation successful
0811 12:15:16.965761 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:15:16.967515 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:16.947121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43619 (local address 127.7.39.2:47113)
0811 12:15:16.948018 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:16.948072 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:16.949785 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:16.950442 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:16.950450 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:16.950988 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:16.951617 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:16.951637 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:16.953042 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:16.953045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:16.955131 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:16.955145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:16.956161 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:16.956166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:16.956380 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:16.957134 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:16.957190 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:16.959629 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:16.962862 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:16.962872 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:16.962892 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:16.963502 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:16.963867 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:16.963872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:16.963875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:16.964139 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:16.964668 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:16.964676 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:16.965140 (+ 464us) client_negotiation.cc:769] Sending connection context
0811 12:15:16.965489 (+ 349us) client_negotiation.cc:240] Negotiation successful
0811 12:15:16.966108 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":288,"threads_started":1}
I20260811 12:15:16.969134 8819 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43619
I20260811 12:15:16.969463 8819 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:16.970268 8819 heartbeater.cc:511] Master 127.7.39.62:43619 requested a full tablet report, sending...
I20260811 12:15:16.971915 8280 ts_manager.cc:194] Registered new tserver with Master: c30e45720dfe418a9c779e25631f17ba (127.7.39.2:37313)
I20260811 12:15:16.973078 8280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:47113
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:15:17.000557 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43619
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:40919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:17.334089 8828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:17.334424 8828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:17.334549 8828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:17.341974 8828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:17.342124 8828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:17.342278 8828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:17.352574 8828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:40919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43619
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:17.354038 8828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:17.355706 8828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:17.369716 8837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:17.369742 8841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:17.369807 8838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:17.374828 8828 server_base.cc:1034] running on GCE node
I20260811 12:15:17.375550 8828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:17.376613 8828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:17.377882 8828 hybrid_clock.cc:648] HybridClock initialized: now 1786450517377818 us; error 45 us; skew 500 ppm
I20260811 12:15:17.378281 8828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:17 dist-test-slave-rmhg krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450517, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:17.381765 8828 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:15:17.383316 8828 webserver.cc:460] Webserver started at http://127.7.39.3:33783/ using document root <none> and password file <none>
I20260811 12:15:17.384104 8828 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:17.384240 8828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:17.384577 8828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:17.387789 8828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "8f53b1d7a0614894a2770ec63090abeb"
format_stamp: "Formatted at 2026-08-11 12:15:17 on dist-test-slave-rmhg"
server_key: "ed5a3648f5b82a5458a85ecb7630940d"
server_key_iv: "124bb715ae3d873260f9a1e31bd78d8a"
server_key_version: "encryptionkey@0"
I20260811 12:15:17.388654 8828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "8f53b1d7a0614894a2770ec63090abeb"
format_stamp: "Formatted at 2026-08-11 12:15:17 on dist-test-slave-rmhg"
server_key: "ed5a3648f5b82a5458a85ecb7630940d"
server_key_iv: "124bb715ae3d873260f9a1e31bd78d8a"
server_key_version: "encryptionkey@0"
I20260811 12:15:17.394385 8828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 12:15:17.398097 8858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:17.399501 8828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:17.399668 8828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "8f53b1d7a0614894a2770ec63090abeb"
format_stamp: "Formatted at 2026-08-11 12:15:17 on dist-test-slave-rmhg"
server_key: "ed5a3648f5b82a5458a85ecb7630940d"
server_key_iv: "124bb715ae3d873260f9a1e31bd78d8a"
server_key_version: "encryptionkey@0"
I20260811 12:15:17.399881 8828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:17.414086 8828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:17.418037 8828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:17.418327 8828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:17.419303 8828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:17.420678 8828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:17.420758 8828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:17.420876 8828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:17.420936 8828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:17.453642 8828 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:32903
I20260811 12:15:17.453876 9035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:32903 every 8 connection(s)
I20260811 12:15:17.455003 8828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:17.461272 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8828
I20260811 12:15:17.461522 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:17.462041 7324 external_mini_cluster.cc:1428] Setting key c7701c62df92007e728274e15c1abe27
Aug 11 12:15:17 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450517, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:17.472893 8594 heartbeater.cc:503] Master 127.7.39.62:43619 was elected leader, sending a full tablet report...
I20260811 12:15:17.482483 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.457666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:50097 (local address 127.7.39.62:43619)
0811 12:15:17.457834 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:17.457839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:17.459948 (+ 2109us) server_negotiation.cc:407] Connection header received
0811 12:15:17.463203 (+ 3255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:17.463209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:17.463306 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:17.463395 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:17.465874 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.466811 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.468654 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.469353 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.469472 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.474506 (+ 5034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:17.474511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:17.474514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:17.474551 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:17.479035 (+ 4484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:17.480047 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:17.480052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:17.480055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:17.480252 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:17.480783 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:17.480787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:17.480789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:17.481069 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:17.481237 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:17.482029 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 12:15:17.482228 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:15:17.483747 9039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.458306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43619 (local address 127.7.39.3:50097)
0811 12:15:17.459642 (+ 1336us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:17.459688 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:17.462883 (+ 3195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:17.463768 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:17.463779 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:17.464704 (+ 925us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:17.465695 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.465720 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.466971 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.466975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.468492 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.468506 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.469719 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.469730 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.470018 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.470685 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:17.470710 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:17.474193 (+ 3483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:17.479256 (+ 5063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:17.479265 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:17.479284 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:17.479893 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:17.480342 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:17.480345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:17.480347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:17.480679 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:17.481227 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:17.481234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:17.481651 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 12:15:17.481900 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 12:15:17.482357 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":502,"threads_started":1}
I20260811 12:15:17.485510 9037 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43619
I20260811 12:15:17.485855 9037 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:17.487025 9037 heartbeater.cc:511] Master 127.7.39.62:43619 requested a full tablet report, sending...
I20260811 12:15:17.488988 8280 ts_manager.cc:194] Registered new tserver with Master: 8f53b1d7a0614894a2770ec63090abeb (127.7.39.3:32903)
I20260811 12:15:17.490146 8280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:50097
I20260811 12:15:17.491993 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:15:17.521028 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.502179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.7.39.62:43619)
0811 12:15:17.502543 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:17.502552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:17.503915 (+ 1363us) server_negotiation.cc:407] Connection header received
0811 12:15:17.504138 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:17.504144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:17.504251 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:17.504359 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:17.507207 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.508200 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.509949 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.510788 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.510921 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.514477 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:17.514483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:17.514486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:17.514532 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:17.517743 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:17.518623 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:17.518628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:17.518630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:17.518816 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:17.519266 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:17.519270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:17.519272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:17.519754 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:17.520076 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:17.520472 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 12:15:17.520712 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:15:17.530827 8280 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:17.542481 8280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:17.578083 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.564193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.7.39.2:37313)
0811 12:15:17.565408 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:17.565416 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:17.566395 (+ 979us) server_negotiation.cc:407] Connection header received
0811 12:15:17.566783 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:17.566792 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:17.567020 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:17.567322 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:17.568957 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.570427 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.574326 (+ 3899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.576095 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.576234 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.576467 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:17.577442 (+ 975us) server_negotiation.cc:299] Negotiation successful
0811 12:15:17.577684 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":859,"threads_started":1}
I20260811 12:15:17.578187 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.561982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.7.39.1:39813)
0811 12:15:17.565023 (+ 3041us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:17.565034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:17.565067 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:15:17.565219 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:17.565227 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:17.565530 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:17.565976 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:17.567816 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.570300 (+ 2484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.574200 (+ 3900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.576094 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.576234 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.576609 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:17.577450 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 12:15:17.577684 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2835,"thread_start_us":2531,"threads_started":1}
I20260811 12:15:17.579207 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.563749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37313 (local address 127.0.0.1:42146)
0811 12:15:17.566127 (+ 2378us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:17.566154 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:17.566558 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:17.567681 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:17.567685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:17.567712 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:17.568740 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.568756 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.570605 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.570610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.574125 (+ 3515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.574142 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.576734 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.576742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.577089 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.577196 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 12:15:17.578334 (+ 1138us) client_negotiation.cc:240] Negotiation successful
0811 12:15:17.578480 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2255,"mutex_wait_us":98,"thread_start_us":2108,"threads_started":1}
I20260811 12:15:17.579577 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.561844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39813 (local address 127.0.0.1:58238)
0811 12:15:17.563282 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:17.563331 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:17.563642 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:17.566032 (+ 2390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:17.566039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:17.566071 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:17.566660 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.566674 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.570462 (+ 3788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.570467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.573957 (+ 3490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.573974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.576902 (+ 2928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.576910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.577230 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.577256 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:15:17.577338 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 12:15:17.577613 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1182,"mutex_wait_us":119,"thread_start_us":887,"threads_started":1}
I20260811 12:15:17.583462 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.566811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.7.39.3:32903)
0811 12:15:17.567845 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:17.567854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:17.571847 (+ 3993us) server_negotiation.cc:407] Connection header received
0811 12:15:17.572206 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:17.572217 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:17.572529 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:17.572809 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:17.574520 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.576008 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.579936 (+ 3928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.581914 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.582054 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.582422 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:17.582676 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 12:15:17.582852 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":667,"threads_started":1}
I20260811 12:15:17.583544 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.566562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:32903 (local address 127.0.0.1:60036)
0811 12:15:17.571685 (+ 5123us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:17.571709 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:17.571893 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:17.573247 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:17.573252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:17.573279 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:17.574316 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.574328 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.576196 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.576202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.579275 (+ 3073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.579295 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.582217 (+ 2922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.582226 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.582447 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.582484 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:15:17.582567 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:15:17.582714 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4966,"mutex_wait_us":1,"thread_start_us":4619,"threads_started":1}
I20260811 12:15:17.584707 8709 tablet_service.cc:1467] Processing CreateTablet for tablet 48205fde66244c88b5dd64d7d6922d00 (DEFAULT_TABLE table=test-table [id=18139b5e19d84ef2ae45199935a9506f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:17.587152 8709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48205fde66244c88b5dd64d7d6922d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:17.586712 8477 tablet_service.cc:1467] Processing CreateTablet for tablet 48205fde66244c88b5dd64d7d6922d00 (DEFAULT_TABLE table=test-table [id=18139b5e19d84ef2ae45199935a9506f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:17.588855 8477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48205fde66244c88b5dd64d7d6922d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:17.588302 8935 tablet_service.cc:1467] Processing CreateTablet for tablet 48205fde66244c88b5dd64d7d6922d00 (DEFAULT_TABLE table=test-table [id=18139b5e19d84ef2ae45199935a9506f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:17.590075 8935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48205fde66244c88b5dd64d7d6922d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:17.607201 9068 tablet_bootstrap.cc:492] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb: Bootstrap starting.
I20260811 12:15:17.612869 9068 tablet_bootstrap.cc:654] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:17.613826 9070 tablet_bootstrap.cc:492] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba: Bootstrap starting.
I20260811 12:15:17.615803 9072 tablet_bootstrap.cc:492] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a: Bootstrap starting.
I20260811 12:15:17.617425 9068 log.cc:824] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:17.618976 9070 tablet_bootstrap.cc:654] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:17.621052 9070 log.cc:824] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:17.621820 9072 tablet_bootstrap.cc:654] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:17.622155 9068 tablet_bootstrap.cc:492] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb: No bootstrap required, opened a new log
I20260811 12:15:17.622598 9068 ts_tablet_manager.cc:1397] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.003s
I20260811 12:15:17.624131 9072 log.cc:824] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:17.625248 9070 tablet_bootstrap.cc:492] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba: No bootstrap required, opened a new log
I20260811 12:15:17.625646 9070 ts_tablet_manager.cc:1397] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.007s
I20260811 12:15:17.628513 9072 tablet_bootstrap.cc:492] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a: No bootstrap required, opened a new log
I20260811 12:15:17.628825 9072 ts_tablet_manager.cc:1397] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260811 12:15:17.636451 9068 raft_consensus.cc:348] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.636966 9068 raft_consensus.cc:374] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:17.637212 9068 raft_consensus.cc:729] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f53b1d7a0614894a2770ec63090abeb, State: Initialized, Role: FOLLOWER
I20260811 12:15:17.638026 9068 consensus_queue.cc:260] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.640024 9037 heartbeater.cc:503] Master 127.7.39.62:43619 was elected leader, sending a full tablet report...
I20260811 12:15:17.639274 9070 raft_consensus.cc:348] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.640715 9070 raft_consensus.cc:374] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:17.640793 9068 ts_tablet_manager.cc:1428] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb: Time spent starting tablet: real 0.018s user 0.008s sys 0.007s
I20260811 12:15:17.641043 9070 raft_consensus.cc:729] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c30e45720dfe418a9c779e25631f17ba, State: Initialized, Role: FOLLOWER
I20260811 12:15:17.641922 9070 consensus_queue.cc:260] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.643858 9072 raft_consensus.cc:348] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.644641 9072 raft_consensus.cc:374] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:17.644801 9072 raft_consensus.cc:729] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f33b631f0ab2497b960e8a7fb828075a, State: Initialized, Role: FOLLOWER
I20260811 12:15:17.645236 8819 heartbeater.cc:503] Master 127.7.39.62:43619 was elected leader, sending a full tablet report...
I20260811 12:15:17.645843 9072 consensus_queue.cc:260] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.646395 9070 ts_tablet_manager.cc:1428] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba: Time spent starting tablet: real 0.021s user 0.014s sys 0.005s
I20260811 12:15:17.648886 9072 ts_tablet_manager.cc:1428] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a: Time spent starting tablet: real 0.020s user 0.015s sys 0.005s
W20260811 12:15:17.690536 8595 tablet.cc:2369] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:15:17.695981 8824 tablet.cc:2369] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:15:17.706717 9038 tablet.cc:2369] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:15:17.834753 9081 raft_consensus.cc:482] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:17.835350 9081 raft_consensus.cc:504] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.838726 9081 leader_election.cc:288] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c30e45720dfe418a9c779e25631f17ba (127.7.39.2:37313), 8f53b1d7a0614894a2770ec63090abeb (127.7.39.3:32903)
I20260811 12:15:17.848347 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.839393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:51395 (local address 127.7.39.2:37313)
0811 12:15:17.839698 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:17.839705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:17.841048 (+ 1343us) server_negotiation.cc:407] Connection header received
0811 12:15:17.841448 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:17.841454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:17.841545 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:17.841666 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:17.842428 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.843326 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.845993 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.847204 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.847363 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.847464 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:17.847777 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 12:15:17.847909 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:15:17.848510 9082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.839217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37313 (local address 127.7.39.1:51395)
0811 12:15:17.840807 (+ 1590us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:17.840826 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:17.841238 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:17.841711 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:17.841717 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:17.841745 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:17.842270 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.842282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.843468 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.843472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.845775 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.845788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.847458 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.847465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.847608 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.847635 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 12:15:17.847722 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:15:17.847865 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1451,"thread_start_us":1028,"threads_started":1}
I20260811 12:15:17.849644 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.840831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:32903 (local address 127.7.39.1:58941)
0811 12:15:17.842010 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:17.842026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:17.842218 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:17.842530 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:17.842534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:17.842559 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:17.843218 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.843229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.844540 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.844544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.846533 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:17.846545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.848764 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.848776 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:17.848973 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.849008 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:15:17.849102 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:15:17.849264 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"thread_start_us":718,"threads_started":1}
I20260811 12:15:17.849700 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.840986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:58941 (local address 127.7.39.3:32903)
0811 12:15:17.841217 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:17.841223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:17.842157 (+ 934us) server_negotiation.cc:407] Connection header received
0811 12:15:17.842358 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:17.842362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:17.842422 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:17.842498 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:17.843400 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.844349 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.846729 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.848060 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.848180 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.848319 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:17.849180 (+ 861us) server_negotiation.cc:299] Negotiation successful
0811 12:15:17.849321 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:15:17.850395 8742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48205fde66244c88b5dd64d7d6922d00" candidate_uuid: "f33b631f0ab2497b960e8a7fb828075a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c30e45720dfe418a9c779e25631f17ba" is_pre_election: true
I20260811 12:15:17.851030 8965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48205fde66244c88b5dd64d7d6922d00" candidate_uuid: "f33b631f0ab2497b960e8a7fb828075a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f53b1d7a0614894a2770ec63090abeb" is_pre_election: true
I20260811 12:15:17.851511 8742 raft_consensus.cc:2454] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f33b631f0ab2497b960e8a7fb828075a in term 0.
I20260811 12:15:17.851811 8965 raft_consensus.cc:2454] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f33b631f0ab2497b960e8a7fb828075a in term 0.
I20260811 12:15:17.852794 8404 leader_election.cc:302] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c30e45720dfe418a9c779e25631f17ba, f33b631f0ab2497b960e8a7fb828075a; no voters:
I20260811 12:15:17.853363 9081 raft_consensus.cc:2781] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:17.853669 9081 raft_consensus.cc:482] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:17.853849 9081 raft_consensus.cc:3037] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:17.856761 9081 raft_consensus.cc:504] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.857674 9081 leader_election.cc:288] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [CANDIDATE]: Term 1 election: Requested vote from peers c30e45720dfe418a9c779e25631f17ba (127.7.39.2:37313), 8f53b1d7a0614894a2770ec63090abeb (127.7.39.3:32903)
I20260811 12:15:17.858201 8965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48205fde66244c88b5dd64d7d6922d00" candidate_uuid: "f33b631f0ab2497b960e8a7fb828075a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f53b1d7a0614894a2770ec63090abeb"
I20260811 12:15:17.858278 8742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48205fde66244c88b5dd64d7d6922d00" candidate_uuid: "f33b631f0ab2497b960e8a7fb828075a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c30e45720dfe418a9c779e25631f17ba"
I20260811 12:15:17.858449 8965 raft_consensus.cc:3037] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:17.858527 8742 raft_consensus.cc:3037] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:17.862166 8965 raft_consensus.cc:2454] T 48205fde66244c88b5dd64d7d6922d00 P 8f53b1d7a0614894a2770ec63090abeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f33b631f0ab2497b960e8a7fb828075a in term 1.
I20260811 12:15:17.862149 8742 raft_consensus.cc:2454] T 48205fde66244c88b5dd64d7d6922d00 P c30e45720dfe418a9c779e25631f17ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f33b631f0ab2497b960e8a7fb828075a in term 1.
I20260811 12:15:17.862664 8404 leader_election.cc:302] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f53b1d7a0614894a2770ec63090abeb, f33b631f0ab2497b960e8a7fb828075a; no voters:
I20260811 12:15:17.863031 9081 raft_consensus.cc:2781] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:17.863721 9081 raft_consensus.cc:686] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [term 1 LEADER]: Becoming Leader. State: Replica: f33b631f0ab2497b960e8a7fb828075a, State: Running, Role: LEADER
I20260811 12:15:17.864416 9081 consensus_queue.cc:237] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } }
I20260811 12:15:17.870652 8277 catalog_manager.cc:5495] T 48205fde66244c88b5dd64d7d6922d00 P f33b631f0ab2497b960e8a7fb828075a reported cstate change: term changed from 0 to 1, leader changed from <none> to f33b631f0ab2497b960e8a7fb828075a (127.7.39.1). New cstate: current_term: 1 leader_uuid: "f33b631f0ab2497b960e8a7fb828075a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f33b631f0ab2497b960e8a7fb828075a" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c30e45720dfe418a9c779e25631f17ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f53b1d7a0614894a2770ec63090abeb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32903 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:15:17 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:17.958158 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:17.934474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.7.39.1:39813)
0811 12:15:17.934831 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:17.934838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:17.935924 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 12:15:17.936095 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:17.936099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:17.936174 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:17.936326 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:17.939145 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.940377 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.942126 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:17.942729 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:17.942854 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:17.948440 (+ 5586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:17.948464 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:17.948484 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:17.948539 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:17.954152 (+ 5613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:17.955148 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:17.955159 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:17.955163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:17.955412 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:17.955881 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:17.955886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:17.955889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:17.956455 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:17.956918 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:17.957080 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 12:15:17.957551 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 12:15:17 dist-test-slave-rmhg krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450517, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:15:18 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450517, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:18.020375 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:18.002741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.7.39.1:39813)
0811 12:15:18.002953 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:18.002960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:18.003137 (+ 177us) server_negotiation.cc:407] Connection header received
0811 12:15:18.003316 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:18.003320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:18.003389 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:18.003468 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:18.004960 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.005910 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.007744 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.008548 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.008661 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:18.013940 (+ 5279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:18.013945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:18.013948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:18.013990 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:18.017438 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.018270 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.018275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.018278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.018488 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.018935 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.018938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.018940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.019329 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:18.019530 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:18.019731 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 12:15:18.019914 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 12:15:18 dist-test-slave-rmhg krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:15:18 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:18.075047 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:18.057675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.7.39.1:39813)
0811 12:15:18.057888 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:18.057896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:18.058361 (+ 465us) server_negotiation.cc:407] Connection header received
0811 12:15:18.058543 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:18.058548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:18.058631 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:18.058726 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:18.060491 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.061364 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.063144 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.063799 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.063903 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:18.068986 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:18.068993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:18.068997 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:18.069035 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:18.072280 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.072946 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.072950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.072953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.073213 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.073499 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.073502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.073505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.073888 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:18.074148 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:18.074289 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 12:15:18.074469 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:15:18.077913 8438 tablet_service.cc:2908] Rejecting scan request for tablet 48205fde66244c88b5dd64d7d6922d00: Uninitialized: clean time has not yet been initialized
I20260811 12:15:18.081611 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8378
I20260811 12:15:18.098273 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:18.082572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58270 (local address 127.7.39.1:39813)
0811 12:15:18.082758 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:18.082765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:18.083204 (+ 439us) server_negotiation.cc:407] Connection header received
0811 12:15:18.083331 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:18.083335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:18.083409 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:18.083489 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:18.085206 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.086046 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.087571 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.087989 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.088091 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:18.091329 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:18.091335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:18.091338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:18.091378 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:18.095114 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.096041 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.096045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.096048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.096184 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.096541 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.096544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.096546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.096892 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:18.097082 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:18.097524 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 12:15:18.097746 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:15:18.099159 8586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:18.193894 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8378
I20260811 12:15:18.221761 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8605
Aug 11 12:15:18 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:15:18.236783 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:18.222546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.7.39.2:37313)
0811 12:15:18.222726 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:18.222731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:18.222751 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:15:18.222895 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:18.222899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:18.222976 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:18.223067 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:18.224364 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.225235 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.226684 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.227120 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.227221 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:18.231359 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:18.231377 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:18.231396 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:18.231442 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:18.234195 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.234883 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.234890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.234892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.235033 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.235354 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.235357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.235359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.235771 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:18.235991 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:18.236286 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:15:18.236463 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:15:18.237645 8807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:18.326099 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8605
I20260811 12:15:18.349680 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8828
Aug 11 12:15:18 dist-test-slave-rmhg krb5kdc[8216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:15:18.364514 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:18.350501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.7.39.3:32903)
0811 12:15:18.350980 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:18.350985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:18.351001 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:15:18.351059 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:18.351062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:18.351134 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:18.351225 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:18.352422 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.353146 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.354479 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:18.354896 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:18.354996 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:18.359253 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:18.359271 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:18.359289 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:18.359335 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:18.362069 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.362767 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.362774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.362776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.362924 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:18.363224 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:18.363227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:18.363229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:18.363594 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:18.363816 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:18.364012 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 12:15:18.364202 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":258,"threads_started":1}
I20260811 12:15:18.365238 9028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:18.454792 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8828
I20260811 12:15:18.477443 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8237
I20260811 12:15:18.478709 8330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:18.589741 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8237
2026-08-11T12:15:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4644 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:18 dist-test-slave-rmhg krb5kdc[9110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:18 dist-test-slave-rmhg krb5kdc[9110](info): set up 2 sockets
Aug 11 12:15:18 dist-test-slave-rmhg krb5kdc[9110](info): commencing operation
krb5kdc: starting...
W20260811 12:15:19.815028 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:19 dist-test-slave-rmhg krb5kdc[9110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:20.026845 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45313
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:45313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:20.368542 9133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:20.368891 9133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:20.369040 9133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:20.376644 9133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:20.376770 9133 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:15:20.376860 9133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:20.376948 9133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:20.377027 9133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:20.377120 9133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:20.389353 9133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:45313
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:20.390720 9133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:20.392165 9133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:20.404520 9140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:20.404627 9151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:20.404457 9142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:20.405630 9133 server_base.cc:1034] running on GCE node
I20260811 12:15:20.406456 9133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:20.407995 9133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:20.409250 9133 hybrid_clock.cc:648] HybridClock initialized: now 1786450520409162 us; error 56 us; skew 500 ppm
I20260811 12:15:20.409602 9133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:20 dist-test-slave-rmhg krb5kdc[9110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450520, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:20.412824 9133 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:20.414484 9133 webserver.cc:460] Webserver started at http://127.7.39.62:34837/ using document root <none> and password file <none>
I20260811 12:15:20.415228 9133 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:20.415338 9133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:20.415664 9133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:20.418826 9133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73"
format_stamp: "Formatted at 2026-08-11 12:15:20 on dist-test-slave-rmhg"
server_key: "b883008d217862a7fc16ee183e221d76"
server_key_iv: "00e4512b0b7a90f2281634b414edd3ab"
server_key_version: "encryptionkey@0"
I20260811 12:15:20.419656 9133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73"
format_stamp: "Formatted at 2026-08-11 12:15:20 on dist-test-slave-rmhg"
server_key: "b883008d217862a7fc16ee183e221d76"
server_key_iv: "00e4512b0b7a90f2281634b414edd3ab"
server_key_version: "encryptionkey@0"
I20260811 12:15:20.425004 9133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:15:20.428853 9171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:20.430380 9133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:20.430552 9133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73"
format_stamp: "Formatted at 2026-08-11 12:15:20 on dist-test-slave-rmhg"
server_key: "b883008d217862a7fc16ee183e221d76"
server_key_iv: "00e4512b0b7a90f2281634b414edd3ab"
server_key_version: "encryptionkey@0"
I20260811 12:15:20.430760 9133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:20.448680 9133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:20.453402 9133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:20.453701 9133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:20.475309 9133 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:45313
I20260811 12:15:20.475332 9260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:45313 every 8 connection(s)
I20260811 12:15:20.476751 9133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:20.478317 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9133
I20260811 12:15:20.478525 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:20.479149 7324 external_mini_cluster.cc:1428] Setting key 92a92aa70b52488dd63cc4321408375c
I20260811 12:15:20.481271 9262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:15:20 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:20.495517 9262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73: Bootstrap starting.
I20260811 12:15:20.499812 9262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:20.501145 9262 log.cc:824] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:20.504853 9262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73: No bootstrap required, opened a new log
I20260811 12:15:20.505208 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:20.481348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43886 (local address 127.7.39.62:45313)
0811 12:15:20.482209 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:20.482226 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:20.482281 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:15:20.484447 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:20.484480 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:20.485070 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:20.485605 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:20.487108 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:20.489245 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:20.490579 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:20.491660 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:20.491858 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:20.496442 (+ 4584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:20.496454 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:20.496474 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:20.496531 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:20.500457 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:20.501247 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:20.501257 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:20.501266 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:20.501490 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:20.501948 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:20.501953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:20.501956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:20.502554 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:20.502855 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:20.503283 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 12:15:20.503753 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":354,"threads_started":1}
I20260811 12:15:20.513545 9262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45313 } }
I20260811 12:15:20.514032 9262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:20.514147 9262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1fb083d4fd4963b5ffbcd7fe50bf73, State: Initialized, Role: FOLLOWER
I20260811 12:15:20.514740 9262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45313 } }
I20260811 12:15:20.514950 9262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:20.515048 9262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:20.515187 9262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:20.517606 9262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45313 } }
I20260811 12:15:20.518092 9262 leader_election.cc:302] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f1fb083d4fd4963b5ffbcd7fe50bf73; no voters:
I20260811 12:15:20.518641 9262 leader_election.cc:288] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:20.518771 9270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:20.519433 9270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [term 1 LEADER]: Becoming Leader. State: Replica: 2f1fb083d4fd4963b5ffbcd7fe50bf73, State: Running, Role: LEADER
I20260811 12:15:20.519977 9270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45313 } }
I20260811 12:15:20.520749 9262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:20.522974 9272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f1fb083d4fd4963b5ffbcd7fe50bf73. Latest consensus state: current_term: 1 leader_uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45313 } } }
I20260811 12:15:20.522945 9271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1fb083d4fd4963b5ffbcd7fe50bf73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45313 } } }
I20260811 12:15:20.523373 9272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:20.523383 9271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:20.524238 9278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:20.532100 9278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:20.544394 9278 catalog_manager.cc:1347] Generated new cluster ID: 2e2a040e865542469ac0c3a9ac95004b
I20260811 12:15:20.544551 9278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:20.555018 9278 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:20.556967 9278 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:20.573179 9278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73: Generated new TSK 0
I20260811 12:15:20.574087 9278 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:15:20.632349 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45313
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:20.980150 9300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:20.980458 9300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:20.980568 9300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:20.987823 9300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:20.987954 9300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:20.988045 9300 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:15:20.988126 9300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:20.998317 9300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:45313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:20.999790 9300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:21.001415 9300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:21.015630 9308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:21.015614 9311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:21.015563 9309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:21.016430 9300 server_base.cc:1034] running on GCE node
I20260811 12:15:21.017107 9300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:21.018103 9300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:21.019330 9300 hybrid_clock.cc:648] HybridClock initialized: now 1786450521019262 us; error 44 us; skew 500 ppm
I20260811 12:15:21.019640 9300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:21 dist-test-slave-rmhg krb5kdc[9110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450521, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:21.029834 9300 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:15:21.031458 9300 webserver.cc:460] Webserver started at http://127.7.39.1:39443/ using document root <none> and password file <none>
I20260811 12:15:21.032218 9300 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:21.032331 9300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:21.032678 9300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:21.035921 9300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "e27f551b56d4476dbc6203461a057897"
format_stamp: "Formatted at 2026-08-11 12:15:21 on dist-test-slave-rmhg"
server_key: "c3ca0d689a4d1019b21d6545a787a6bb"
server_key_iv: "be29ef5102ce5caf8ce7d66109a3abe6"
server_key_version: "encryptionkey@0"
I20260811 12:15:21.036731 9300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "e27f551b56d4476dbc6203461a057897"
format_stamp: "Formatted at 2026-08-11 12:15:21 on dist-test-slave-rmhg"
server_key: "c3ca0d689a4d1019b21d6545a787a6bb"
server_key_iv: "be29ef5102ce5caf8ce7d66109a3abe6"
server_key_version: "encryptionkey@0"
I20260811 12:15:21.042191 9300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:15:21.046033 9328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:21.047521 9300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:21.047722 9300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "e27f551b56d4476dbc6203461a057897"
format_stamp: "Formatted at 2026-08-11 12:15:21 on dist-test-slave-rmhg"
server_key: "c3ca0d689a4d1019b21d6545a787a6bb"
server_key_iv: "be29ef5102ce5caf8ce7d66109a3abe6"
server_key_version: "encryptionkey@0"
I20260811 12:15:21.047950 9300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:21.065435 9300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:21.069729 9300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:21.069993 9300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:21.072825 9300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:21.072919 9300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:21.073048 9300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:21.073128 9300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:15:21 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450521, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:21.096838 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:21.073940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:59793 (local address 127.7.39.62:45313)
0811 12:15:21.074397 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:21.074402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:21.075842 (+ 1440us) server_negotiation.cc:407] Connection header received
0811 12:15:21.077987 (+ 2145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:21.077995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:21.078090 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:21.078180 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:21.080672 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.081918 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.084211 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.084640 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.084762 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:21.089596 (+ 4834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:21.089604 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:21.089609 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:21.089656 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:21.093175 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:21.094263 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:21.094269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:21.094272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:21.094439 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:21.094925 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:21.094931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:21.094934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:21.095244 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:21.095388 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:21.096204 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 12:15:21.096458 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":239,"threads_started":1}
I20260811 12:15:21.098358 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:21.074743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45313 (local address 127.7.39.1:59793)
0811 12:15:21.075681 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:21.075725 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:21.077645 (+ 1920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:21.078777 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:21.078785 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:21.079573 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:21.080461 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:21.080488 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.082138 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.082144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:21.084008 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:21.084024 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.084927 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.084935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:21.085228 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:21.086067 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:21.086104 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:21.089339 (+ 3235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:21.093341 (+ 4002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:21.093356 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:21.093381 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:21.094118 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:21.094552 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:21.094557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:21.094561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:21.094799 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:21.095381 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:21.095389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:21.095786 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 12:15:21.096076 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 12:15:21.096675 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":376,"threads_started":1}
I20260811 12:15:21.114430 9300 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:40319
I20260811 12:15:21.114452 9520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:40319 every 8 connection(s)
I20260811 12:15:21.115826 9300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:21.118575 9521 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45313
I20260811 12:15:21.119040 9521 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:21.119953 9521 heartbeater.cc:511] Master 127.7.39.62:45313 requested a full tablet report, sending...
I20260811 12:15:21.121407 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:21.105981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:35829 (local address 127.7.39.62:45313)
0811 12:15:21.106205 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:21.106212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:21.107224 (+ 1012us) server_negotiation.cc:407] Connection header received
0811 12:15:21.107362 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:21.107366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:21.107435 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:21.107545 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:21.109276 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.110259 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.112305 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.112749 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.112872 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:21.115218 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:21.115224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:21.115228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:21.115264 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:21.118438 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:21.119311 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:21.119315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:21.119318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:21.119461 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:21.119850 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:21.119853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:21.119855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:21.120195 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:21.120310 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:21.120785 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 12:15:21.121037 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:15:21.121498 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:21.106299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45313 (local address 127.7.39.1:35829)
0811 12:15:21.107114 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:21.107131 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:21.107263 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:21.107717 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:21.107722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:21.108480 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:21.109048 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:21.109069 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.110449 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.110453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:21.112129 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:21.112148 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.112961 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.112968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:21.113136 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:21.113909 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:21.113928 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:21.114994 (+ 1066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:21.118582 (+ 3588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:21.118588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:21.118592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:21.119194 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:21.119575 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:21.119580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:21.119583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:21.119750 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:21.120309 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:21.120314 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:21.120512 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 12:15:21.120692 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 12:15:21.121036 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":346,"threads_started":1}
I20260811 12:15:21.122506 9194 ts_manager.cc:194] Registered new tserver with Master: e27f551b56d4476dbc6203461a057897 (127.7.39.1:40319)
I20260811 12:15:21.124491 9194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:59793
I20260811 12:15:21.125387 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9300
I20260811 12:15:21.125603 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:21.126344 7324 external_mini_cluster.cc:1428] Setting key e9e02742b0673a3398374f6f8dad8c91
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:15:21.173297 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45313
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:21.509366 9531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:21.509680 9531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:21.509795 9531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:21.518155 9531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:21.518277 9531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:21.518368 9531 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:15:21.518450 9531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:21.528597 9531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:45313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:21.529987 9531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:21.531317 9531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:21.544770 9540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:21.544843 9543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:21.544869 9541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:21.545830 9531 server_base.cc:1034] running on GCE node
I20260811 12:15:21.546435 9531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:21.547433 9531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:21.548646 9531 hybrid_clock.cc:648] HybridClock initialized: now 1786450521548580 us; error 41 us; skew 500 ppm
I20260811 12:15:21.548941 9531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:21 dist-test-slave-rmhg krb5kdc[9110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450521, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:21.552093 9531 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:15:21.553643 9531 webserver.cc:460] Webserver started at http://127.7.39.2:39201/ using document root <none> and password file <none>
I20260811 12:15:21.554373 9531 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:21.554477 9531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:21.554788 9531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:21.557811 9531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "c3d32f5538cc47de81e6c5ea9be9a72c"
format_stamp: "Formatted at 2026-08-11 12:15:21 on dist-test-slave-rmhg"
server_key: "10dc22a3003de556292b0c945e9e27cb"
server_key_iv: "b523217c7c20d0143dfea1c2be26f270"
server_key_version: "encryptionkey@0"
I20260811 12:15:21.558611 9531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "c3d32f5538cc47de81e6c5ea9be9a72c"
format_stamp: "Formatted at 2026-08-11 12:15:21 on dist-test-slave-rmhg"
server_key: "10dc22a3003de556292b0c945e9e27cb"
server_key_iv: "b523217c7c20d0143dfea1c2be26f270"
server_key_version: "encryptionkey@0"
I20260811 12:15:21.563849 9531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:15:21.567680 9555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:21.569413 9531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:21.569588 9531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "c3d32f5538cc47de81e6c5ea9be9a72c"
format_stamp: "Formatted at 2026-08-11 12:15:21 on dist-test-slave-rmhg"
server_key: "10dc22a3003de556292b0c945e9e27cb"
server_key_iv: "b523217c7c20d0143dfea1c2be26f270"
server_key_version: "encryptionkey@0"
I20260811 12:15:21.569810 9531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:21.597114 9531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:21.601653 9531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:21.601935 9531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:21.604820 9531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:21.604926 9531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:21.605065 9531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:21.605142 9531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:15:21 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450521, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:21.633944 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:21.605396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:49345 (local address 127.7.39.62:45313)
0811 12:15:21.605646 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:21.605653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:21.607364 (+ 1711us) server_negotiation.cc:407] Connection header received
0811 12:15:21.610173 (+ 2809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:21.610180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:21.610271 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:21.610403 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:21.613117 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.614400 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.618741 (+ 4341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.619196 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.619314 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:21.624800 (+ 5486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:21.624809 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:21.624819 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:21.624884 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:21.629208 (+ 4324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:21.630370 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:21.630376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:21.630380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:21.630554 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:21.631222 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:21.631228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:21.631231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:21.631692 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:21.632861 (+ 1169us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:21.633223 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 12:15:21.633519 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 12:15:21.636142 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:21.606126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45313 (local address 127.7.39.2:49345)
0811 12:15:21.607102 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:21.607166 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:21.609842 (+ 2676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:21.611165 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:21.611179 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:21.611957 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:21.612876 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:21.612904 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.614703 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.614709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:21.617247 (+ 2538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:21.617266 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.619441 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.619449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:21.619737 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:21.621343 (+ 1606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:21.621368 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:21.624438 (+ 3070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:21.629437 (+ 4999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:21.629458 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:21.629481 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:21.630227 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:21.630700 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:21.630706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:21.630713 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:21.631051 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:21.632036 (+ 985us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:21.632046 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:21.632652 (+ 606us) client_negotiation.cc:769] Sending connection context
0811 12:15:21.633013 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 12:15:21.633716 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":403,"threads_started":1}
I20260811 12:15:21.666126 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:21.646915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:52265 (local address 127.7.39.62:45313)
0811 12:15:21.647127 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:21.647133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:21.648255 (+ 1122us) server_negotiation.cc:407] Connection header received
0811 12:15:21.648578 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:21.648586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:21.648671 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:21.648744 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:21.650543 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.651787 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.654251 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.654921 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.655023 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:21.658230 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:21.658239 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:21.658243 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:21.658293 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:21.662133 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:21.663304 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:21.663310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:21.663314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:21.663487 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:21.664000 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:21.664006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:21.664010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:21.664539 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:21.664684 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:21.665499 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 12:15:21.665755 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:15:21.667464 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:21.646533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45313 (local address 127.7.39.2:52265)
0811 12:15:21.648112 (+ 1579us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:21.648137 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:21.648408 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:21.648979 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:21.648984 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:21.649789 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:21.650327 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:21.650346 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.652094 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.652100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:21.654016 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:21.654034 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:21.655384 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:21.655390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:21.655647 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:21.656472 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:21.656498 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:21.657939 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:21.662331 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:21.662338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:21.662342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:21.663124 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:21.663651 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:21.663657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:21.663664 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:21.663858 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:21.664784 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:21.664789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:21.665170 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 12:15:21.666449 (+ 1279us) client_negotiation.cc:240] Negotiation successful
0811 12:15:21.666881 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1390,"thread_start_us":1206,"threads_started":1}
I20260811 12:15:21.670183 9531 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:40259
I20260811 12:15:21.670202 9751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:40259 every 8 connection(s)
I20260811 12:15:21.671766 9531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:21.674420 9752 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45313
I20260811 12:15:21.674739 9752 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:21.675122 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9531
I20260811 12:15:21.675341 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:21.675623 9752 heartbeater.cc:511] Master 127.7.39.62:45313 requested a full tablet report, sending...
I20260811 12:15:21.675897 7324 external_mini_cluster.cc:1428] Setting key 3af608892a17cf7c030126be74b40de1
I20260811 12:15:21.677361 9194 ts_manager.cc:194] Registered new tserver with Master: c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259)
I20260811 12:15:21.678544 9194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:49345
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:15:21.726547 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45313
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:22.060511 9756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:22.060788 9756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:22.060887 9756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:22.068228 9756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:22.068343 9756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:22.068426 9756 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:15:22.068516 9756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:22.078761 9756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:45313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:22.080073 9756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:22.081485 9756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:22.094923 9765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:22.094972 9762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:22.094950 9768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:22.095707 9756 server_base.cc:1034] running on GCE node
I20260811 12:15:22.096299 9756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:22.097340 9756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:22.098556 9756 hybrid_clock.cc:648] HybridClock initialized: now 1786450522098492 us; error 39 us; skew 500 ppm
I20260811 12:15:22.098855 9756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:22 dist-test-slave-rmhg krb5kdc[9110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450522, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:22.102187 9756 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:15:22.103766 9756 webserver.cc:460] Webserver started at http://127.7.39.3:38601/ using document root <none> and password file <none>
I20260811 12:15:22.104487 9756 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:22.104586 9756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:22.104897 9756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:22.107950 9756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "67d9ccbdb06543999a779c6173c31148"
format_stamp: "Formatted at 2026-08-11 12:15:22 on dist-test-slave-rmhg"
server_key: "e6a62ef68e88fe171d0a8dc6e24e5ffa"
server_key_iv: "2db777ee70ed4958586fd52bdfbac95d"
server_key_version: "encryptionkey@0"
I20260811 12:15:22.108759 9756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "67d9ccbdb06543999a779c6173c31148"
format_stamp: "Formatted at 2026-08-11 12:15:22 on dist-test-slave-rmhg"
server_key: "e6a62ef68e88fe171d0a8dc6e24e5ffa"
server_key_iv: "2db777ee70ed4958586fd52bdfbac95d"
server_key_version: "encryptionkey@0"
I20260811 12:15:22.114343 9756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 12:15:22.118172 9776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:22.119653 9756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:22.119820 9756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "67d9ccbdb06543999a779c6173c31148"
format_stamp: "Formatted at 2026-08-11 12:15:22 on dist-test-slave-rmhg"
server_key: "e6a62ef68e88fe171d0a8dc6e24e5ffa"
server_key_iv: "2db777ee70ed4958586fd52bdfbac95d"
server_key_version: "encryptionkey@0"
I20260811 12:15:22.120042 9756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:22.127909 9521 heartbeater.cc:503] Master 127.7.39.62:45313 was elected leader, sending a full tablet report...
I20260811 12:15:22.141357 9756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:22.146088 9756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:22.146361 9756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:22.149258 9756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:22.149366 9756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:22.149497 9756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:22.149575 9756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:15:22 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450522, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:22.175530 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.149814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:57243 (local address 127.7.39.62:45313)
0811 12:15:22.150017 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.150023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.151785 (+ 1762us) server_negotiation.cc:407] Connection header received
0811 12:15:22.154120 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.154127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.154221 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.154336 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:22.156629 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.157575 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.161224 (+ 3649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.161899 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.162061 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.167539 (+ 5478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:22.167549 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:22.167554 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:22.167607 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:22.171057 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:22.172307 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:22.172314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:22.172317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:22.172495 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:22.173196 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:22.173202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:22.173205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:22.173573 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:22.173756 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.174787 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.175098 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:15:22.176374 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.150596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45313 (local address 127.7.39.3:57243)
0811 12:15:22.151621 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:22.151662 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:22.153778 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:22.154762 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:22.154770 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:22.155582 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:22.156425 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.156447 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.158691 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.158695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.160996 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.161013 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.162262 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.162272 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.162512 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.163407 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:22.163460 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:22.167187 (+ 3727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:22.171258 (+ 4071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:22.171275 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:22.171297 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:22.172115 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:22.172642 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:22.172648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:22.172652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:22.173029 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:22.173852 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:22.173859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:22.174322 (+ 463us) client_negotiation.cc:769] Sending connection context
0811 12:15:22.174613 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 12:15:22.175121 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":425,"threads_started":1}
I20260811 12:15:22.196187 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.182884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45313 (local address 127.7.39.3:56267)
0811 12:15:22.183500 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:22.183521 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:22.183666 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:22.184020 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:22.184024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:22.184522 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:22.184813 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.184824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.186046 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.186051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.187087 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.187100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.187847 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.187854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.188003 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.188689 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:22.188708 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:22.189954 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:22.192932 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:22.192938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:22.192942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:22.193596 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:22.194044 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:22.194053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:22.194056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:22.194238 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:22.194841 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:22.194846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:22.195092 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 12:15:22.195311 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 12:15:22.195700 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":333,"threads_started":1}
I20260811 12:15:22.196969 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.182922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:56267 (local address 127.7.39.62:45313)
0811 12:15:22.183088 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.183092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.183617 (+ 525us) server_negotiation.cc:407] Connection header received
0811 12:15:22.183789 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.183795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.183873 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.183959 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:22.184969 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.185901 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.187231 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.187659 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.187770 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.190147 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:22.190152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:22.190156 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:22.190198 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:22.192810 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:22.193774 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:22.193780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:22.193784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:22.193913 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:22.194353 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:22.194358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:22.194362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:22.194739 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:22.194881 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.196537 (+ 1656us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.196722 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:15:22.201442 9756 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:33289
I20260811 12:15:22.201463 9965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:33289 every 8 connection(s)
I20260811 12:15:22.202720 9756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:22.204774 9967 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45313
I20260811 12:15:22.205017 9967 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:22.205710 9967 heartbeater.cc:511] Master 127.7.39.62:45313 requested a full tablet report, sending...
I20260811 12:15:22.207168 9194 ts_manager.cc:194] Registered new tserver with Master: 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289)
I20260811 12:15:22.208031 9194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:57243
I20260811 12:15:22.209750 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9756
I20260811 12:15:22.209916 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:22.210340 7324 external_mini_cluster.cc:1428] Setting key cc8c04dca4a2d43d3720a7ecc86475d0
I20260811 12:15:22.213265 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:15:22 dist-test-slave-rmhg krb5kdc[9110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450522, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:15:22 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450522, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:22.266083 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.251308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.7.39.62:45313)
0811 12:15:22.251573 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.251580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.252020 (+ 440us) server_negotiation.cc:407] Connection header received
0811 12:15:22.252180 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.252185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.252270 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.252354 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:22.253857 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.254600 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.255963 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.256395 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.256498 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.260534 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:22.260540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:22.260543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:22.260583 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:22.263253 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:22.263960 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:22.263965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:22.263967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:22.264107 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:22.264516 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:22.264520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:22.264522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:22.264828 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:22.265067 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.265470 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.265715 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:15:22.270359 9194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:22.277586 9194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:22.326603 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.291760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.7.39.3:33289)
0811 12:15:22.315177 (+ 23417us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.315188 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.318634 (+ 3446us) server_negotiation.cc:407] Connection header received
0811 12:15:22.318777 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.318785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.319017 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.319278 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:22.320029 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.321663 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.324260 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.325371 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.325497 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.325767 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.326077 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.326261 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23171,"thread_start_us":23446,"threads_started":1}
I20260811 12:15:22.326694 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.291246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33289 (local address 127.0.0.1:48628)
0811 12:15:22.318496 (+ 27250us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:22.318512 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:22.318633 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:22.319252 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:22.319256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:22.319281 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:22.319799 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.319808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.321857 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.321862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.324123 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.324137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.325579 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.325586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.325804 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.325862 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 12:15:22.326017 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 12:15:22.326232 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27140,"mutex_wait_us":26,"thread_start_us":25348,"threads_started":1}
I20260811 12:15:22.326992 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.290669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52474 (local address 127.7.39.2:40259)
0811 12:15:22.318185 (+ 27516us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.318193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.318230 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:15:22.318340 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.318348 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.318642 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.318864 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:22.319618 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.321003 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.323802 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.325412 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.325549 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.325776 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.326525 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.326709 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27342,"thread_start_us":27215,"threads_started":1}
I20260811 12:15:22.327195 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.290467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:40259 (local address 127.0.0.1:52474)
0811 12:15:22.316092 (+ 25625us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:22.316149 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:22.316359 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:22.318877 (+ 2518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:22.318885 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:22.318914 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:22.319413 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.319430 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.321174 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.321179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.323637 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.323651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.326003 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.326010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.326199 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.326295 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 12:15:22.326384 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:15:22.326819 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25288,"mutex_wait_us":368,"thread_start_us":27754,"threads_started":1}
I20260811 12:15:22.327553 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.291150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40319 (local address 127.0.0.1:58916)
0811 12:15:22.318222 (+ 27072us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:22.318246 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:22.318404 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:22.319162 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:22.319166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:22.319190 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:22.319793 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.319807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.321247 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.321252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.323864 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.323881 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.325968 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.325974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.326193 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.326293 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 12:15:22.326384 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:15:22.326548 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26906,"mutex_wait_us":140,"thread_start_us":26728,"threads_started":1}
I20260811 12:15:22.328059 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.290988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.7.39.1:40319)
0811 12:15:22.317726 (+ 26738us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.317736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.318471 (+ 735us) server_negotiation.cc:407] Connection header received
0811 12:15:22.318571 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.318578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.318782 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.318956 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:22.320019 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.321017 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.324021 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.325485 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.325607 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.325778 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.327589 (+ 1811us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.327780 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26558,"thread_start_us":26414,"threads_started":1}
I20260811 12:15:22.330884 9639 tablet_service.cc:1467] Processing CreateTablet for tablet 7d3216064b8f40aa9b0a2dddb23efaca (DEFAULT_TABLE table=test-table [id=0b3f2ec4b8c140999e6294c253086766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:22.330983 9861 tablet_service.cc:1467] Processing CreateTablet for tablet 7d3216064b8f40aa9b0a2dddb23efaca (DEFAULT_TABLE table=test-table [id=0b3f2ec4b8c140999e6294c253086766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:22.331503 9402 tablet_service.cc:1467] Processing CreateTablet for tablet 7d3216064b8f40aa9b0a2dddb23efaca (DEFAULT_TABLE table=test-table [id=0b3f2ec4b8c140999e6294c253086766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:22.332396 9639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d3216064b8f40aa9b0a2dddb23efaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:22.332495 9861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d3216064b8f40aa9b0a2dddb23efaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:22.332803 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d3216064b8f40aa9b0a2dddb23efaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:22.345413 10000 tablet_bootstrap.cc:492] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Bootstrap starting.
I20260811 12:15:22.348349 10002 tablet_bootstrap.cc:492] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Bootstrap starting.
I20260811 12:15:22.350669 10003 tablet_bootstrap.cc:492] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Bootstrap starting.
I20260811 12:15:22.350934 10000 tablet_bootstrap.cc:654] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:22.353483 10000 log.cc:824] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:22.354288 10002 tablet_bootstrap.cc:654] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:22.355445 10003 tablet_bootstrap.cc:654] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:22.356539 10002 log.cc:824] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:22.357311 10003 log.cc:824] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:22.358503 10000 tablet_bootstrap.cc:492] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: No bootstrap required, opened a new log
I20260811 12:15:22.358903 10000 ts_tablet_manager.cc:1397] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260811 12:15:22.360533 10002 tablet_bootstrap.cc:492] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: No bootstrap required, opened a new log
I20260811 12:15:22.360986 10002 ts_tablet_manager.cc:1397] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260811 12:15:22.362106 10003 tablet_bootstrap.cc:492] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: No bootstrap required, opened a new log
I20260811 12:15:22.362594 10003 ts_tablet_manager.cc:1397] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260811 12:15:22.369674 10000 raft_consensus.cc:348] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.370328 10000 raft_consensus.cc:374] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:22.370452 10000 raft_consensus.cc:729] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d32f5538cc47de81e6c5ea9be9a72c, State: Initialized, Role: FOLLOWER
I20260811 12:15:22.371235 10000 consensus_queue.cc:260] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.374168 9752 heartbeater.cc:503] Master 127.7.39.62:45313 was elected leader, sending a full tablet report...
I20260811 12:15:22.374915 10000 ts_tablet_manager.cc:1428] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Time spent starting tablet: real 0.016s user 0.009s sys 0.006s
I20260811 12:15:22.378430 10003 raft_consensus.cc:348] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.379101 10003 raft_consensus.cc:374] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:22.379264 10003 raft_consensus.cc:729] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d9ccbdb06543999a779c6173c31148, State: Initialized, Role: FOLLOWER
I20260811 12:15:22.380244 10003 consensus_queue.cc:260] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.381639 10002 raft_consensus.cc:348] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.382246 10002 raft_consensus.cc:374] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:22.382396 10002 raft_consensus.cc:729] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e27f551b56d4476dbc6203461a057897, State: Initialized, Role: FOLLOWER
I20260811 12:15:22.383236 10002 consensus_queue.cc:260] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.387326 10002 ts_tablet_manager.cc:1428] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Time spent starting tablet: real 0.026s user 0.009s sys 0.013s
I20260811 12:15:22.388612 9967 heartbeater.cc:503] Master 127.7.39.62:45313 was elected leader, sending a full tablet report...
I20260811 12:15:22.389279 10003 ts_tablet_manager.cc:1428] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Time spent starting tablet: real 0.027s user 0.018s sys 0.008s
W20260811 12:15:22.424057 9753 tablet.cc:2369] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:15:22.454296 9968 tablet.cc:2369] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:15:22.578886 10014 raft_consensus.cc:482] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:22.579217 10014 raft_consensus.cc:504] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.581125 10014 leader_election.cc:288] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259), 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289)
I20260811 12:15:22.587906 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.581808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:43939 (local address 127.7.39.2:40259)
0811 12:15:22.582040 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.582047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.582662 (+ 615us) server_negotiation.cc:407] Connection header received
0811 12:15:22.582826 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.582829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.582888 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.582962 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:22.583602 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.584311 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.586199 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.586977 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.587103 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.587214 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.587514 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.587650 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 12:15:22.588104 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.581535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:40259 (local address 127.7.39.1:43939)
0811 12:15:22.582543 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:22.582558 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:22.582686 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:22.583004 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:22.583008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:22.583081 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:22.583447 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.583456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.584456 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.584460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.586025 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.586042 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.587205 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.587211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.587355 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.587380 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:15:22.587463 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:15:22.587594 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":22,"spinlock_wait_cycles":35072,"thread_start_us":723,"threads_started":1}
I20260811 12:15:22.588785 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.582318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:39555 (local address 127.7.39.3:33289)
0811 12:15:22.582573 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.582578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.582607 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:15:22.582674 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.582678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.582738 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.582813 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:22.583428 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.584448 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.586325 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.587538 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.587667 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.587806 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.588401 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.588503 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 12:15:22.589653 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.581460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33289 (local address 127.7.39.1:39555)
0811 12:15:22.582300 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:22.582323 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:22.582492 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:22.582856 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:22.582860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:22.582888 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:22.583282 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.583293 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.584590 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.584594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.586133 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.586146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.588012 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.588020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.588203 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.588236 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:15:22.588976 (+ 740us) client_negotiation.cc:240] Negotiation successful
0811 12:15:22.589128 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":447,"threads_started":1}
I20260811 12:15:22.589282 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d3216064b8f40aa9b0a2dddb23efaca" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" is_pre_election: true
I20260811 12:15:22.590071 9677 raft_consensus.cc:2454] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 0.
I20260811 12:15:22.590780 9891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d3216064b8f40aa9b0a2dddb23efaca" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d9ccbdb06543999a779c6173c31148" is_pre_election: true
I20260811 12:15:22.591037 9336 leader_election.cc:302] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d32f5538cc47de81e6c5ea9be9a72c, e27f551b56d4476dbc6203461a057897; no voters:
I20260811 12:15:22.591398 9891 raft_consensus.cc:2454] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 0.
I20260811 12:15:22.591645 10014 raft_consensus.cc:2781] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:22.591895 10014 raft_consensus.cc:482] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:22.592078 10014 raft_consensus.cc:3037] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:22.595880 10014 raft_consensus.cc:504] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.596731 10014 leader_election.cc:288] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 election: Requested vote from peers c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259), 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289)
I20260811 12:15:22.597255 9891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d3216064b8f40aa9b0a2dddb23efaca" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d9ccbdb06543999a779c6173c31148"
I20260811 12:15:22.597285 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d3216064b8f40aa9b0a2dddb23efaca" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c"
I20260811 12:15:22.597489 9891 raft_consensus.cc:3037] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:22.597489 9677 raft_consensus.cc:3037] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:22.601020 9891 raft_consensus.cc:2454] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 1.
I20260811 12:15:22.601616 9330 leader_election.cc:302] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d9ccbdb06543999a779c6173c31148, e27f551b56d4476dbc6203461a057897; no voters:
I20260811 12:15:22.602035 9677 raft_consensus.cc:2454] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 1.
I20260811 12:15:22.602128 10014 raft_consensus.cc:2781] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:22.602854 10014 raft_consensus.cc:686] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 1 LEADER]: Becoming Leader. State: Replica: e27f551b56d4476dbc6203461a057897, State: Running, Role: LEADER
I20260811 12:15:22.603557 10014 consensus_queue.cc:237] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.608805 9192 catalog_manager.cc:5495] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 reported cstate change: term changed from 0 to 1, leader changed from <none> to e27f551b56d4476dbc6203461a057897 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } health_report { overall_health: UNKNOWN } } }
W20260811 12:15:22.618991 9523 tablet.cc:2369] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:15:22.691042 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.683974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58918 (local address 127.7.39.1:40319)
0811 12:15:22.684214 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.684221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.684502 (+ 281us) server_negotiation.cc:407] Connection header received
0811 12:15:22.684956 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.684960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.685127 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.685285 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:22.685738 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.686692 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.688203 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.688773 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.688869 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.689638 (+ 769us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:22.690430 (+ 792us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:22.690544 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.690655 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.690732 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:15:22.707378 9891 raft_consensus.cc:1261] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Refusing update from remote peer e27f551b56d4476dbc6203461a057897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:22.708851 10018 consensus_queue.cc:1035] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 12:15:22.709360 9677 raft_consensus.cc:1261] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Refusing update from remote peer e27f551b56d4476dbc6203461a057897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:22.710377 10018 consensus_queue.cc:1035] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:22.725132 10024 mvcc.cc:205] Tried to move back new op lower bound from 7317301340975316992 to 7317301340588421120. Current Snapshot: MvccSnapshot[applied={T|T < 7317301340975316992}]
I20260811 12:15:22.754313 9194 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:22.754823 9194 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:22.759765 9194 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73: Sending DeleteTablet for 3 replicas of tablet 7d3216064b8f40aa9b0a2dddb23efaca
I20260811 12:15:22.761492 9402 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d3216064b8f40aa9b0a2dddb23efaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:22 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:58916
I20260811 12:15:22.761523 9639 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d3216064b8f40aa9b0a2dddb23efaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:22 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:52474
I20260811 12:15:22.761952 9861 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d3216064b8f40aa9b0a2dddb23efaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:22 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:48628
I20260811 12:15:22.767011 9194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:22.768110 9194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:22.776650 10041 tablet_replica.cc:330] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: stopping tablet replica
I20260811 12:15:22.777570 10041 raft_consensus.cc:2229] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:22.778792 10042 tablet_replica.cc:330] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: stopping tablet replica
I20260811 12:15:22.778683 9401 tablet_service.cc:1467] Processing CreateTablet for tablet fe2cbb338c63448eb89d342c35f9d7dc (DEFAULT_TABLE table=test-table [id=539e37168ab34a638ca0606658f88086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:22.779529 10039 tablet_replica.cc:330] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: stopping tablet replica
I20260811 12:15:22.779531 10042 raft_consensus.cc:2229] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:22.779906 9401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2cbb338c63448eb89d342c35f9d7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:22.780468 10042 raft_consensus.cc:2258] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:22.780377 10039 raft_consensus.cc:2229] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:15:22.781497 10041 raft_consensus.cc:2258] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:22.782635 9861 tablet_service.cc:1467] Processing CreateTablet for tablet fe2cbb338c63448eb89d342c35f9d7dc (DEFAULT_TABLE table=test-table [id=539e37168ab34a638ca0606658f88086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:22.783380 9861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2cbb338c63448eb89d342c35f9d7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:22.784940 10042 ts_tablet_manager.cc:1905] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:22.787212 10039 raft_consensus.cc:2258] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:22.782473 9639 tablet_service.cc:1467] Processing CreateTablet for tablet fe2cbb338c63448eb89d342c35f9d7dc (DEFAULT_TABLE table=test-table [id=539e37168ab34a638ca0606658f88086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:22.789777 10039 ts_tablet_manager.cc:1905] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:22.789829 9639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2cbb338c63448eb89d342c35f9d7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:22.793721 10003 tablet_bootstrap.cc:492] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: Bootstrap starting.
I20260811 12:15:22.796485 10003 tablet_bootstrap.cc:654] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:22.796768 10041 ts_tablet_manager.cc:1905] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:22.797864 10000 tablet_bootstrap.cc:492] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: Bootstrap starting.
I20260811 12:15:22.801314 10002 tablet_bootstrap.cc:492] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: Bootstrap starting.
I20260811 12:15:22.801805 10003 tablet_bootstrap.cc:492] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: No bootstrap required, opened a new log
I20260811 12:15:22.802047 10003 ts_tablet_manager.cc:1397] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260811 12:15:22.802091 10000 tablet_bootstrap.cc:654] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:22.803117 10003 raft_consensus.cc:348] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.803424 10003 raft_consensus.cc:374] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:22.803629 10003 raft_consensus.cc:729] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d9ccbdb06543999a779c6173c31148, State: Initialized, Role: FOLLOWER
I20260811 12:15:22.803923 10003 consensus_queue.cc:260] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.804158 10002 tablet_bootstrap.cc:654] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:22.805047 10042 ts_tablet_manager.cc:1918] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:15:22.806502 10003 ts_tablet_manager.cc:1428] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 12:15:22.809762 10000 tablet_bootstrap.cc:492] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: No bootstrap required, opened a new log
I20260811 12:15:22.810142 10000 ts_tablet_manager.cc:1397] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260811 12:15:22.810987 10039 ts_tablet_manager.cc:1918] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:15:22.811316 10039 log.cc:1196] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/7d3216064b8f40aa9b0a2dddb23efaca
I20260811 12:15:22.805464 10042 log.cc:1196] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/7d3216064b8f40aa9b0a2dddb23efaca
I20260811 12:15:22.811707 10000 raft_consensus.cc:348] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.811968 10000 raft_consensus.cc:374] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:22.812054 10039 ts_tablet_manager.cc:1939] T 7d3216064b8f40aa9b0a2dddb23efaca P e27f551b56d4476dbc6203461a057897: Deleting consensus metadata
I20260811 12:15:22.812189 10000 raft_consensus.cc:729] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d32f5538cc47de81e6c5ea9be9a72c, State: Initialized, Role: FOLLOWER
I20260811 12:15:22.812737 10000 consensus_queue.cc:260] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.813990 10002 tablet_bootstrap.cc:492] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: No bootstrap required, opened a new log
I20260811 12:15:22.814291 10002 ts_tablet_manager.cc:1397] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.001s
I20260811 12:15:22.814466 9173 catalog_manager.cc:4841] TS e27f551b56d4476dbc6203461a057897 (127.7.39.1:40319): tablet 7d3216064b8f40aa9b0a2dddb23efaca (table test-table [id=0b3f2ec4b8c140999e6294c253086766]) successfully deleted
I20260811 12:15:22.815089 10000 ts_tablet_manager.cc:1428] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 12:15:22.815747 10002 raft_consensus.cc:348] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.816108 10002 raft_consensus.cc:374] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:22.816359 10002 raft_consensus.cc:729] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e27f551b56d4476dbc6203461a057897, State: Initialized, Role: FOLLOWER
I20260811 12:15:22.817004 10002 consensus_queue.cc:260] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.818806 10042 ts_tablet_manager.cc:1939] T 7d3216064b8f40aa9b0a2dddb23efaca P 67d9ccbdb06543999a779c6173c31148: Deleting consensus metadata
I20260811 12:15:22.818960 10002 ts_tablet_manager.cc:1428] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260811 12:15:22.820863 10041 ts_tablet_manager.cc:1918] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:15:22.820968 9175 catalog_manager.cc:4841] TS 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289): tablet 7d3216064b8f40aa9b0a2dddb23efaca (table test-table [id=0b3f2ec4b8c140999e6294c253086766]) successfully deleted
I20260811 12:15:22.821249 10041 log.cc:1196] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/7d3216064b8f40aa9b0a2dddb23efaca
I20260811 12:15:22.822155 10041 ts_tablet_manager.cc:1939] T 7d3216064b8f40aa9b0a2dddb23efaca P c3d32f5538cc47de81e6c5ea9be9a72c: Deleting consensus metadata
I20260811 12:15:22.824193 9177 catalog_manager.cc:4841] TS c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259): tablet 7d3216064b8f40aa9b0a2dddb23efaca (table test-table [id=0b3f2ec4b8c140999e6294c253086766]) successfully deleted
I20260811 12:15:22.901487 10014 raft_consensus.cc:482] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:22.901902 10014 raft_consensus.cc:504] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.902843 10014 leader_election.cc:288] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259), 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289)
I20260811 12:15:22.903275 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2cbb338c63448eb89d342c35f9d7dc" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" is_pre_election: true
I20260811 12:15:22.903570 9677 raft_consensus.cc:2454] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 0.
I20260811 12:15:22.903674 9891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2cbb338c63448eb89d342c35f9d7dc" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d9ccbdb06543999a779c6173c31148" is_pre_election: true
I20260811 12:15:22.903944 9891 raft_consensus.cc:2454] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 0.
I20260811 12:15:22.904141 9336 leader_election.cc:302] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d32f5538cc47de81e6c5ea9be9a72c, e27f551b56d4476dbc6203461a057897; no voters:
I20260811 12:15:22.904526 10014 raft_consensus.cc:2781] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:22.904614 10014 raft_consensus.cc:482] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:22.904731 10014 raft_consensus.cc:3037] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:22.907711 10014 raft_consensus.cc:504] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.908531 10014 leader_election.cc:288] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 election: Requested vote from peers c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259), 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289)
I20260811 12:15:22.909087 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2cbb338c63448eb89d342c35f9d7dc" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c"
I20260811 12:15:22.909361 9677 raft_consensus.cc:3037] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:22.909337 9891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2cbb338c63448eb89d342c35f9d7dc" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d9ccbdb06543999a779c6173c31148"
I20260811 12:15:22.909654 9891 raft_consensus.cc:3037] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:22.912389 9677 raft_consensus.cc:2454] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 1.
I20260811 12:15:22.913002 9336 leader_election.cc:302] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d32f5538cc47de81e6c5ea9be9a72c, e27f551b56d4476dbc6203461a057897; no voters:
I20260811 12:15:22.913237 9891 raft_consensus.cc:2454] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 1.
I20260811 12:15:22.913401 10014 raft_consensus.cc:2781] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:22.913841 10014 raft_consensus.cc:686] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 1 LEADER]: Becoming Leader. State: Replica: e27f551b56d4476dbc6203461a057897, State: Running, Role: LEADER
I20260811 12:15:22.914140 10014 consensus_queue.cc:237] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:22.917460 9192 catalog_manager.cc:5495] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 reported cstate change: term changed from 0 to 1, leader changed from <none> to e27f551b56d4476dbc6203461a057897 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:15:22 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450520, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:22.978348 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.956488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45313 (local address 127.0.0.1:43900)
0811 12:15:22.957747 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:22.957779 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:22.957964 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:22.958715 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:22.958720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:22.959890 (+ 1170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:22.960321 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.960338 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.961972 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.961978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.964256 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:22.964275 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.965389 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.965397 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:22.965613 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.966494 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:22.966519 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:22.970165 (+ 3646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:22.974618 (+ 4453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:22.974637 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:22.974646 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:22.975520 (+ 874us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:22.975978 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:22.975985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:22.975989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:22.976280 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:22.976958 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:22.976968 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:22.977234 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 12:15:22.977484 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 12:15:22.977838 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1011,"thread_start_us":764,"threads_started":1}
I20260811 12:15:22.978391 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:22.956512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.7.39.62:45313)
0811 12:15:22.958404 (+ 1892us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:22.958409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:22.958430 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:15:22.958486 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:22.958490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:22.958547 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:22.958643 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:22.960516 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.961818 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.964431 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:22.965200 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:22.965304 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:22.970468 (+ 5164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:22.970474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:22.970477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:22.970509 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:22.974418 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:22.975690 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:22.975695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:22.975698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:22.975836 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:22.976395 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:22.976400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:22.976402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:22.976792 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:22.976902 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:22.977562 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 12:15:22.977966 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":1689,"thread_start_us":1422,"threads_started":1}
I20260811 12:15:22.985852 9192 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:43900:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:15:22.987660 9192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:22.995673 9401 tablet_service.cc:1467] Processing CreateTablet for tablet eff7f604b97643a389c767a1250cea58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=67ffd2ec10f74ca7a2ad4a53b2411f05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:15:22.995942 9639 tablet_service.cc:1467] Processing CreateTablet for tablet eff7f604b97643a389c767a1250cea58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=67ffd2ec10f74ca7a2ad4a53b2411f05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:15:22.996363 9401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eff7f604b97643a389c767a1250cea58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:22.996824 9639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eff7f604b97643a389c767a1250cea58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:23.001566 9861 tablet_service.cc:1467] Processing CreateTablet for tablet eff7f604b97643a389c767a1250cea58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=67ffd2ec10f74ca7a2ad4a53b2411f05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:15:23.002257 9861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eff7f604b97643a389c767a1250cea58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:23.006680 10002 tablet_bootstrap.cc:492] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: Bootstrap starting.
I20260811 12:15:23.007861 10000 tablet_bootstrap.cc:492] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: Bootstrap starting.
I20260811 12:15:23.009774 10002 tablet_bootstrap.cc:654] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:23.010865 10000 tablet_bootstrap.cc:654] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:23.011034 10003 tablet_bootstrap.cc:492] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: Bootstrap starting.
I20260811 12:15:23.013119 10003 tablet_bootstrap.cc:654] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:23.024825 10003 tablet_bootstrap.cc:492] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: No bootstrap required, opened a new log
I20260811 12:15:23.025115 10002 tablet_bootstrap.cc:492] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: No bootstrap required, opened a new log
I20260811 12:15:23.025220 10003 ts_tablet_manager.cc:1397] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.001s
I20260811 12:15:23.025498 10002 ts_tablet_manager.cc:1397] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.005s
I20260811 12:15:23.025825 10000 tablet_bootstrap.cc:492] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: No bootstrap required, opened a new log
I20260811 12:15:23.026638 10000 ts_tablet_manager.cc:1397] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.007s
I20260811 12:15:23.026527 10003 raft_consensus.cc:348] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.026628 10002 raft_consensus.cc:348] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.026938 10003 raft_consensus.cc:374] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:23.027004 10002 raft_consensus.cc:374] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:23.027127 10003 raft_consensus.cc:729] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d9ccbdb06543999a779c6173c31148, State: Initialized, Role: FOLLOWER
I20260811 12:15:23.027158 10002 raft_consensus.cc:729] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e27f551b56d4476dbc6203461a057897, State: Initialized, Role: FOLLOWER
I20260811 12:15:23.027484 10002 consensus_queue.cc:260] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.027508 10003 consensus_queue.cc:260] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.028036 10014 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.028398 10014 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:23.028304 10000 raft_consensus.cc:348] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.028255 10003 ts_tablet_manager.cc:1428] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260811 12:15:23.028249 10012 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.028643 10000 raft_consensus.cc:374] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:23.028767 10000 raft_consensus.cc:729] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d32f5538cc47de81e6c5ea9be9a72c, State: Initialized, Role: FOLLOWER
I20260811 12:15:23.028836 10012 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:23.029238 10000 consensus_queue.cc:260] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.030040 10008 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.030565 10008 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:23.031246 10002 ts_tablet_manager.cc:1428] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 12:15:23.032073 10000 ts_tablet_manager.cc:1428] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 12:15:23.248378 10014 raft_consensus.cc:482] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:23.248596 10014 raft_consensus.cc:504] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.249344 10014 leader_election.cc:288] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259), 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289)
I20260811 12:15:23.249971 9891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eff7f604b97643a389c767a1250cea58" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d9ccbdb06543999a779c6173c31148" is_pre_election: true
I20260811 12:15:23.249971 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eff7f604b97643a389c767a1250cea58" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" is_pre_election: true
I20260811 12:15:23.250263 9677 raft_consensus.cc:2454] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 0.
I20260811 12:15:23.250404 9891 raft_consensus.cc:2454] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 0.
I20260811 12:15:23.250753 9336 leader_election.cc:302] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d32f5538cc47de81e6c5ea9be9a72c, e27f551b56d4476dbc6203461a057897; no voters:
I20260811 12:15:23.251153 10014 raft_consensus.cc:2781] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:23.251441 10014 raft_consensus.cc:482] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:23.251618 10014 raft_consensus.cc:3037] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:23.254554 10014 raft_consensus.cc:504] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.255245 10014 leader_election.cc:288] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 election: Requested vote from peers c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259), 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289)
I20260811 12:15:23.255725 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eff7f604b97643a389c767a1250cea58" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c"
I20260811 12:15:23.255998 9677 raft_consensus.cc:3037] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:23.255920 9891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eff7f604b97643a389c767a1250cea58" candidate_uuid: "e27f551b56d4476dbc6203461a057897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d9ccbdb06543999a779c6173c31148"
I20260811 12:15:23.256301 9891 raft_consensus.cc:3037] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:23.259166 9677 raft_consensus.cc:2454] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 1.
I20260811 12:15:23.259754 9891 raft_consensus.cc:2454] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e27f551b56d4476dbc6203461a057897 in term 1.
I20260811 12:15:23.259824 9336 leader_election.cc:302] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d32f5538cc47de81e6c5ea9be9a72c, e27f551b56d4476dbc6203461a057897; no voters:
I20260811 12:15:23.260264 10014 raft_consensus.cc:2781] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:23.260442 10014 raft_consensus.cc:686] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [term 1 LEADER]: Becoming Leader. State: Replica: e27f551b56d4476dbc6203461a057897, State: Running, Role: LEADER
I20260811 12:15:23.260876 10014 consensus_queue.cc:237] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } }
I20260811 12:15:23.263413 10018 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: TxnStatusTablet state changed. Reason: New leader e27f551b56d4476dbc6203461a057897. Latest consensus state: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.263873 10018 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:15:23.265093 9194 catalog_manager.cc:5495] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 reported cstate change: term changed from 0 to 1, leader changed from <none> to e27f551b56d4476dbc6203461a057897 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } health_report { overall_health: UNKNOWN } } }
I20260811 12:15:23.269555 10069 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:15:23.269882 10069 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:15:23.315011 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.306737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40319 (local address 127.0.0.1:58924)
0811 12:15:23.307356 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:23.307377 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:23.307562 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:23.308009 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:23.308014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:23.308032 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:15:23.308337 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:23.308352 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.310172 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.310178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:23.311585 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:23.311605 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.313104 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.313112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:23.313336 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.313419 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:15:23.314036 (+ 617us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:15:23.314139 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 12:15:23.314294 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 12:15:23.314494 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368}
I20260811 12:15:23.315142 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.306287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.7.39.1:40319)
0811 12:15:23.307327 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:23.307334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:23.307464 (+ 130us) server_negotiation.cc:407] Connection header received
0811 12:15:23.307773 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:23.307777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:23.307852 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:23.307941 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:23.308499 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.309972 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.311797 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.312882 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.313023 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.313769 (+ 746us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:23.313910 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:23.313998 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:23.314418 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 12:15:23.314523 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":663,"threads_started":1}
I20260811 12:15:23.322129 9891 raft_consensus.cc:1261] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Refusing update from remote peer e27f551b56d4476dbc6203461a057897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:23.322386 9677 raft_consensus.cc:1261] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Refusing update from remote peer e27f551b56d4476dbc6203461a057897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:23.322940 10014 consensus_queue.cc:1035] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:23.323400 10018 consensus_queue.cc:1035] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:23.330323 10008 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: TxnStatusTablet state changed. Reason: New leader e27f551b56d4476dbc6203461a057897. Latest consensus state: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.330957 10008 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:23.334566 10012 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: TxnStatusTablet state changed. Reason: New leader e27f551b56d4476dbc6203461a057897. Latest consensus state: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.335165 10012 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:23.338459 10008 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.338796 10008 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P c3d32f5538cc47de81e6c5ea9be9a72c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:23.338749 10018 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.339162 10018 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:15:23.339440 10018 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.339890 10018 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P e27f551b56d4476dbc6203461a057897: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:15:23.347081 10012 tablet_replica.cc:439] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e27f551b56d4476dbc6203461a057897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27f551b56d4476dbc6203461a057897" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40319 } } peers { permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 } } peers { permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 } } }
I20260811 12:15:23.347457 10012 tablet_replica.cc:442] T eff7f604b97643a389c767a1250cea58 P 67d9ccbdb06543999a779c6173c31148: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:23.348418 10014 consensus_queue.cc:1035] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d32f5538cc47de81e6c5ea9be9a72c" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 40259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:23.355273 10018 consensus_queue.cc:1035] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d9ccbdb06543999a779c6173c31148" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:23.386052 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.373894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:35439 (local address 127.7.39.1:40319)
0811 12:15:23.374110 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:23.374118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:23.379765 (+ 5647us) server_negotiation.cc:407] Connection header received
0811 12:15:23.380001 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:23.380005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:23.380067 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:23.380144 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:23.380871 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.381881 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.383819 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.384572 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.384701 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.385273 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:23.385388 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:23.385466 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:23.385617 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 12:15:23.385730 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":67}
I20260811 12:15:23.386302 10096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.373773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40319 (local address 127.7.39.1:35439)
0811 12:15:23.379627 (+ 5854us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:23.379652 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:23.379864 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:23.380176 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:23.380180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:23.380196 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:15:23.380701 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:23.380716 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.382036 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.382042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:23.383658 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:23.383675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.384789 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.384795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:23.384980 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.385052 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:15:23.385487 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:15:23.385498 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 12:15:23.385571 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:15:23.385751 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5577,"mutex_wait_us":1,"thread_start_us":5306,"threads_started":1}
I20260811 12:15:23.480460 9192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:23.480815 9192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:23.484977 9192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f1fb083d4fd4963b5ffbcd7fe50bf73: Sending DeleteTablet for 3 replicas of tablet fe2cbb338c63448eb89d342c35f9d7dc
I20260811 12:15:23.486253 9861 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2cbb338c63448eb89d342c35f9d7dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:23 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:48628
I20260811 12:15:23.487202 9639 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2cbb338c63448eb89d342c35f9d7dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:23 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:52474
I20260811 12:15:23.487823 9402 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2cbb338c63448eb89d342c35f9d7dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:23 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:58916
I20260811 12:15:23.494050 10103 tablet_replica.cc:330] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: stopping tablet replica
I20260811 12:15:23.494349 10103 raft_consensus.cc:2229] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:23.494817 10103 raft_consensus.cc:2258] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:23.502636 10103 ts_tablet_manager.cc:1905] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:23.503491 10106 tablet_replica.cc:330] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: stopping tablet replica
I20260811 12:15:23.503753 10106 raft_consensus.cc:2229] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:23.504284 10106 raft_consensus.cc:2258] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:23.506680 10107 tablet_replica.cc:330] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: stopping tablet replica
I20260811 12:15:23.506963 10107 raft_consensus.cc:2229] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:15:23.507845 10107 raft_consensus.cc:2258] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:23.509532 10107 ts_tablet_manager.cc:1905] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:23.510767 10103 ts_tablet_manager.cc:1918] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:23.511023 10103 log.cc:1196] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/fe2cbb338c63448eb89d342c35f9d7dc
I20260811 12:15:23.511767 10103 ts_tablet_manager.cc:1939] T fe2cbb338c63448eb89d342c35f9d7dc P 67d9ccbdb06543999a779c6173c31148: Deleting consensus metadata
I20260811 12:15:23.513222 9175 catalog_manager.cc:4841] TS 67d9ccbdb06543999a779c6173c31148 (127.7.39.3:33289): tablet fe2cbb338c63448eb89d342c35f9d7dc (table test-table [id=539e37168ab34a638ca0606658f88086]) successfully deleted
I20260811 12:15:23.522284 10106 ts_tablet_manager.cc:1905] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:23.525275 10107 ts_tablet_manager.cc:1918] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:23.525547 10107 log.cc:1196] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/fe2cbb338c63448eb89d342c35f9d7dc
I20260811 12:15:23.526260 10107 ts_tablet_manager.cc:1939] T fe2cbb338c63448eb89d342c35f9d7dc P e27f551b56d4476dbc6203461a057897: Deleting consensus metadata
I20260811 12:15:23.527596 9173 catalog_manager.cc:4841] TS e27f551b56d4476dbc6203461a057897 (127.7.39.1:40319): tablet fe2cbb338c63448eb89d342c35f9d7dc (table test-table [id=539e37168ab34a638ca0606658f88086]) successfully deleted
I20260811 12:15:23.531350 10106 ts_tablet_manager.cc:1918] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:23.531617 10106 log.cc:1196] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/fe2cbb338c63448eb89d342c35f9d7dc
I20260811 12:15:23.532089 10106 ts_tablet_manager.cc:1939] T fe2cbb338c63448eb89d342c35f9d7dc P c3d32f5538cc47de81e6c5ea9be9a72c: Deleting consensus metadata
I20260811 12:15:23.533258 9177 catalog_manager.cc:4841] TS c3d32f5538cc47de81e6c5ea9be9a72c (127.7.39.2:40259): tablet fe2cbb338c63448eb89d342c35f9d7dc (table test-table [id=539e37168ab34a638ca0606658f88086]) successfully deleted
Aug 11 12:15:23 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450522, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:23.545113 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.510391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58934 (local address 127.7.39.1:40319)
0811 12:15:23.510710 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:23.510720 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:23.518190 (+ 7470us) server_negotiation.cc:407] Connection header received
0811 12:15:23.518762 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:23.518768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:23.518857 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:23.518942 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:23.522799 (+ 3857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.524228 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.526649 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.527379 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.527496 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.535206 (+ 7710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:23.535228 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:23.535252 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:23.535306 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:23.541140 (+ 5834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.542293 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.542309 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.542312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.542542 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.542998 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.543004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.543008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.543592 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:23.543976 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:23.544256 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 12:15:23.544567 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
W20260811 12:15:23.546343 9514 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58934
I20260811 12:15:23.568219 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.550117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43910 (local address 127.7.39.62:45313)
0811 12:15:23.551659 (+ 1542us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:23.551664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:23.551681 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:15:23.551737 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:23.551741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:23.551809 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:23.551904 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:23.554224 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.555419 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.557061 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.557821 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.557940 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.561098 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:23.561104 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:23.561108 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:23.561143 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:23.564970 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.565815 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.565822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.565825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.566074 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.566512 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.566516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.566518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.566879 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:23.567131 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:23.567523 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 12:15:23.567801 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1362,"thread_start_us":1187,"threads_started":1}
W20260811 12:15:23.568887 9192 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43910
I20260811 12:15:23.571805 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9300
I20260811 12:15:23.590999 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.572927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.7.39.1:40319)
0811 12:15:23.573452 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:23.573457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:23.573516 (+ 59us) server_negotiation.cc:407] Connection header received
0811 12:15:23.573804 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:23.573808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:23.573875 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:23.573951 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:23.576167 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.576910 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.578998 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.579562 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.579692 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.583683 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:23.583688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:23.583692 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:23.583733 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:23.587686 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.588723 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.588727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.588729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.588938 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.589427 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.589431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.589433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.589817 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:23.590228 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:23.590431 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 12:15:23.590694 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 12:15:23.591702 9514 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58950
W20260811 12:15:23.592334 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9300: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:23.592679 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9300
I20260811 12:15:23.626160 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9531
Aug 11 12:15:23 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450522, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:15:23.641857 10117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.626916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52490 (local address 127.7.39.2:40259)
0811 12:15:23.627369 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:23.627374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:23.627390 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:15:23.627446 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:23.627450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:23.627521 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:23.627621 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:23.629124 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.629904 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.631197 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.631648 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.631743 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.635885 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:23.635905 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:23.635924 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:23.635970 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:23.639200 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.639887 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.639896 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.639898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.640119 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.640435 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.640438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.640440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.640810 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:23.641037 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:23.641332 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:15:23.641529 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":241,"threads_started":1}
W20260811 12:15:23.642616 9745 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52490
W20260811 12:15:23.643419 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9531: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:23.643575 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9531
I20260811 12:15:23.679980 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9756
Aug 11 12:15:23 dist-test-slave-rmhg krb5kdc[9110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450522, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:15:23.695611 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:23.680799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48630 (local address 127.7.39.3:33289)
0811 12:15:23.681311 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:23.681316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:23.681333 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:15:23.681390 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:23.681393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:23.681467 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:23.681560 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:23.682780 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.683478 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.684691 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:23.685102 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:23.685323 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:23.689642 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:23.689664 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:23.689684 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:23.689727 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:23.692799 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.693660 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.693670 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.693672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.693854 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:23.694221 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:23.694225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:23.694227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:23.694625 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:23.694881 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:23.695107 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 12:15:23.695325 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":269,"threads_started":1}
W20260811 12:15:23.696352 9959 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48630
W20260811 12:15:23.697190 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9756: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:23.697366 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9756
I20260811 12:15:23.717530 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9133
I20260811 12:15:23.718765 9253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:23.839236 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9133
2026-08-11T12:15:23Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5253 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:23 dist-test-slave-rmhg krb5kdc[10129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:23 dist-test-slave-rmhg krb5kdc[10129](info): set up 2 sockets
Aug 11 12:15:23 dist-test-slave-rmhg krb5kdc[10129](info): commencing operation
krb5kdc: starting...
W20260811 12:15:25.057480 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:25 dist-test-slave-rmhg krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:25.276387 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:37675
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:25.614873 10148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:25.615188 10148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:25.615286 10148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:25.622586 10148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:25.622714 10148 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:15:25.622756 10148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:25.622820 10148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:25.622870 10148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:25.622933 10148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:25.633183 10148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:37675
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:37675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:25.634538 10148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:25.635924 10148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:25.648393 10155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:25.648356 10159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:25.648432 10156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:25.649315 10148 server_base.cc:1034] running on GCE node
I20260811 12:15:25.650117 10148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:25.651592 10148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:25.652824 10148 hybrid_clock.cc:648] HybridClock initialized: now 1786450525652757 us; error 32 us; skew 500 ppm
I20260811 12:15:25.653137 10148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:25 dist-test-slave-rmhg krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450525, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:25.656313 10148 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:25.658082 10148 webserver.cc:460] Webserver started at http://127.7.39.62:35163/ using document root <none> and password file <none>
I20260811 12:15:25.658804 10148 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:25.658892 10148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:25.659204 10148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:25.662343 10148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "9eed0a07d56a4236a6aae0ed6908ddf7"
format_stamp: "Formatted at 2026-08-11 12:15:25 on dist-test-slave-rmhg"
server_key: "ea276699fce7df9564243f2bb09ee051"
server_key_iv: "05c24ae5c982104b56180a12f6e7ad30"
server_key_version: "encryptionkey@0"
I20260811 12:15:25.663148 10148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "9eed0a07d56a4236a6aae0ed6908ddf7"
format_stamp: "Formatted at 2026-08-11 12:15:25 on dist-test-slave-rmhg"
server_key: "ea276699fce7df9564243f2bb09ee051"
server_key_iv: "05c24ae5c982104b56180a12f6e7ad30"
server_key_version: "encryptionkey@0"
I20260811 12:15:25.668382 10148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:15:25.672015 10169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:25.673401 10148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:25.673591 10148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "9eed0a07d56a4236a6aae0ed6908ddf7"
format_stamp: "Formatted at 2026-08-11 12:15:25 on dist-test-slave-rmhg"
server_key: "ea276699fce7df9564243f2bb09ee051"
server_key_iv: "05c24ae5c982104b56180a12f6e7ad30"
server_key_version: "encryptionkey@0"
I20260811 12:15:25.673792 10148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:25.686632 10148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:25.691067 10148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:25.691315 10148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:25.712603 10148 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:37675
I20260811 12:15:25.712628 10259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:37675 every 8 connection(s)
I20260811 12:15:25.714181 10148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:25.717809 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10148
I20260811 12:15:25.718053 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:25.718308 10261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:25.718711 7324 external_mini_cluster.cc:1428] Setting key c00d4cb3d6cdf5bf4e0e15019ab4ca7b
I20260811 12:15:25.733371 10261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: Bootstrap starting.
Aug 11 12:15:25 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:25.737504 10261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:25.738952 10261 log.cc:824] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:25.742681 10261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: No bootstrap required, opened a new log
I20260811 12:15:25.744032 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:25.720741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45512 (local address 127.7.39.62:37675)
0811 12:15:25.721650 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:25.721662 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:25.721711 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:15:25.723338 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:25.723363 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:25.723783 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:25.724145 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:25.726147 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:25.727501 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:25.729309 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:25.730054 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:25.730209 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:25.735109 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:25.735122 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:25.735144 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:25.735210 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:25.739142 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:25.740029 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:25.740036 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:25.740043 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:25.740222 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:25.740635 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:25.740640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:25.740643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:25.741427 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:25.741722 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:25.742147 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 12:15:25.742631 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":438,"threads_started":1}
I20260811 12:15:25.750730 10261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eed0a07d56a4236a6aae0ed6908ddf7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37675 } }
I20260811 12:15:25.751120 10261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:25.751240 10261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eed0a07d56a4236a6aae0ed6908ddf7, State: Initialized, Role: FOLLOWER
I20260811 12:15:25.751794 10261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eed0a07d56a4236a6aae0ed6908ddf7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37675 } }
I20260811 12:15:25.752025 10261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:25.752154 10261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:25.752305 10261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:25.754599 10261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eed0a07d56a4236a6aae0ed6908ddf7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37675 } }
I20260811 12:15:25.755087 10261 leader_election.cc:302] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9eed0a07d56a4236a6aae0ed6908ddf7; no voters:
I20260811 12:15:25.755635 10261 leader_election.cc:288] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:25.755746 10268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:25.756377 10268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [term 1 LEADER]: Becoming Leader. State: Replica: 9eed0a07d56a4236a6aae0ed6908ddf7, State: Running, Role: LEADER
I20260811 12:15:25.756906 10268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eed0a07d56a4236a6aae0ed6908ddf7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37675 } }
I20260811 12:15:25.757704 10261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:25.759795 10270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9eed0a07d56a4236a6aae0ed6908ddf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eed0a07d56a4236a6aae0ed6908ddf7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37675 } } }
I20260811 12:15:25.760196 10270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:25.765722 10271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9eed0a07d56a4236a6aae0ed6908ddf7. Latest consensus state: current_term: 1 leader_uuid: "9eed0a07d56a4236a6aae0ed6908ddf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eed0a07d56a4236a6aae0ed6908ddf7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37675 } } }
I20260811 12:15:25.766217 10271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7 [sys.catalog]: This master's current role is: LEADER
W20260811 12:15:25.771292 10291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:15:25.771415 10291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:15:25.771522 10280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:25.773176 10280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:25.782200 10280 catalog_manager.cc:1347] Generated new cluster ID: 0068b58f1495475d9be48eebbe932d34
I20260811 12:15:25.782339 10280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:25.794453 10280 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:25.795876 10280 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:25.804380 10280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: Generated new TSK 0
I20260811 12:15:25.805326 10280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:15:25.869320 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37675
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:26.236846 10296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:26.237207 10296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:26.237335 10296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:26.245327 10296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:26.245445 10296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:26.245540 10296 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:15:26.245594 10296 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:15:26.245664 10296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:26.255900 10296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:37675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:26.257345 10296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:26.258756 10296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:26.271910 10303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:26.271950 10304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:26.271973 10307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:26.272635 10296 server_base.cc:1034] running on GCE node
I20260811 12:15:26.273344 10296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:26.274283 10296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:26.275488 10296 hybrid_clock.cc:648] HybridClock initialized: now 1786450526275427 us; error 52 us; skew 500 ppm
I20260811 12:15:26.275835 10296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:26 dist-test-slave-rmhg krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:26.285940 10296 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:15:26.287699 10296 webserver.cc:460] Webserver started at http://127.7.39.1:44293/ using document root <none> and password file <none>
I20260811 12:15:26.288450 10296 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:26.288587 10296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:26.288908 10296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:26.292128 10296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "d97463d1a263450fa63aa714db09e877"
format_stamp: "Formatted at 2026-08-11 12:15:26 on dist-test-slave-rmhg"
server_key: "70bcef11e8e7b44b59c41866ebf4d70c"
server_key_iv: "844e078e73034b23d6f51f01bc9a208e"
server_key_version: "encryptionkey@0"
I20260811 12:15:26.292933 10296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "d97463d1a263450fa63aa714db09e877"
format_stamp: "Formatted at 2026-08-11 12:15:26 on dist-test-slave-rmhg"
server_key: "70bcef11e8e7b44b59c41866ebf4d70c"
server_key_iv: "844e078e73034b23d6f51f01bc9a208e"
server_key_version: "encryptionkey@0"
I20260811 12:15:26.298606 10296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:15:26.302609 10318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:26.304332 10296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:26.304515 10296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "d97463d1a263450fa63aa714db09e877"
format_stamp: "Formatted at 2026-08-11 12:15:26 on dist-test-slave-rmhg"
server_key: "70bcef11e8e7b44b59c41866ebf4d70c"
server_key_iv: "844e078e73034b23d6f51f01bc9a208e"
server_key_version: "encryptionkey@0"
I20260811 12:15:26.304724 10296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:26.346222 10296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:26.350693 10296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:26.350971 10296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:26.353835 10296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:26.353960 10296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:26.354089 10296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:26.354172 10296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:15:26 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:26.380553 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:26.354823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:59463 (local address 127.7.39.62:37675)
0811 12:15:26.355306 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:26.355311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:26.356689 (+ 1378us) server_negotiation.cc:407] Connection header received
0811 12:15:26.359187 (+ 2498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:26.359193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:26.359314 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:26.359417 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:26.361504 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.362583 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.365570 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.366060 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.366196 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:26.371379 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:26.371387 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:26.371394 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:26.371446 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:26.375263 (+ 3817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:26.376558 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:26.376565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:26.376569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:26.376783 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:26.377503 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:26.377509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:26.377512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:26.377934 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:26.378085 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:26.379821 (+ 1736us) server_negotiation.cc:299] Negotiation successful
0811 12:15:26.380124 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":259,"threads_started":1}
I20260811 12:15:26.382022 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:26.355572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37675 (local address 127.7.39.1:59463)
0811 12:15:26.356485 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:26.356537 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:26.358834 (+ 2297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:26.359720 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:26.359729 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:26.360335 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:26.361271 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:26.361304 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.362819 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.362826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:26.365367 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:26.365386 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.366354 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.366364 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:26.366693 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:26.367613 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:26.367656 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:26.370989 (+ 3333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:26.375500 (+ 4511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:26.375513 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:26.375538 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:26.376354 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:26.376934 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:26.376939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:26.376943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:26.377332 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:26.378211 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:26.378223 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:26.378834 (+ 611us) client_negotiation.cc:769] Sending connection context
0811 12:15:26.379206 (+ 372us) client_negotiation.cc:240] Negotiation successful
0811 12:15:26.380172 (+ 966us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":390,"threads_started":1}
I20260811 12:15:26.404353 10296 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:44545
I20260811 12:15:26.404356 10515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:44545 every 8 connection(s)
I20260811 12:15:26.405663 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:26.390084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37675 (local address 127.7.39.1:53379)
0811 12:15:26.390711 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:26.390730 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:26.390902 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:26.391452 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:26.391456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:26.392178 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:26.392707 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:26.392720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.393904 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.393909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:26.395629 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:26.395647 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.396466 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.396482 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:26.396636 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:26.397377 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:26.397408 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:26.398792 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:26.402597 (+ 3805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:26.402604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:26.402608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:26.403235 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:26.403724 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:26.403730 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:26.403735 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:26.403925 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:26.404631 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:26.404635 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:26.404809 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 12:15:26.404945 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 12:15:26.405285 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":340,"threads_started":1}
I20260811 12:15:26.405745 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:26.389798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:53379 (local address 127.7.39.62:37675)
0811 12:15:26.389980 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:26.389985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:26.390841 (+ 856us) server_negotiation.cc:407] Connection header received
0811 12:15:26.391054 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:26.391058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:26.391159 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:26.391262 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:26.392881 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.393762 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.395780 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.396249 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.396369 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:26.399048 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:26.399055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:26.399060 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:26.399107 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:26.402424 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:26.403411 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:26.403417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:26.403421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:26.403593 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:26.404065 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:26.404071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:26.404075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:26.404506 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:26.404693 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:26.405078 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 12:15:26.405365 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:15:26.406267 10296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:26.406761 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10296
I20260811 12:15:26.406948 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:26.407763 7324 external_mini_cluster.cc:1428] Setting key 5a96c53bc2cd9e6173ee324cc1defd26
I20260811 12:15:26.409821 10516 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37675
I20260811 12:15:26.410090 10516 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:26.410971 10516 heartbeater.cc:511] Master 127.7.39.62:37675 requested a full tablet report, sending...
I20260811 12:15:26.413275 10194 ts_manager.cc:194] Registered new tserver with Master: d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545)
I20260811 12:15:26.414772 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:59463
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:15:26.457260 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37675
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:26.799678 10521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:26.799976 10521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:26.800092 10521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:26.807518 10521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:26.807662 10521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:26.807750 10521 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:15:26.807804 10521 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:15:26.807876 10521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:26.818066 10521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:37675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:26.819371 10521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:26.820698 10521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:26.833827 10535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:26.834064 10530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:26.834470 10532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:26.834637 10521 server_base.cc:1034] running on GCE node
I20260811 12:15:26.835273 10521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:26.836194 10521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:26.837390 10521 hybrid_clock.cc:648] HybridClock initialized: now 1786450526837338 us; error 45 us; skew 500 ppm
I20260811 12:15:26.837705 10521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:26 dist-test-slave-rmhg krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:26.840878 10521 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:15:26.842401 10521 webserver.cc:460] Webserver started at http://127.7.39.2:32907/ using document root <none> and password file <none>
I20260811 12:15:26.843096 10521 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:26.843209 10521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:26.843634 10521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:26.846781 10521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "bee4528070b24536ae7bc91b362a1edb"
format_stamp: "Formatted at 2026-08-11 12:15:26 on dist-test-slave-rmhg"
server_key: "5c7bc524db519cebcf9dd7927d1a14e0"
server_key_iv: "0d66e7e988224ccf5e1b209d847da255"
server_key_version: "encryptionkey@0"
I20260811 12:15:26.847527 10521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "bee4528070b24536ae7bc91b362a1edb"
format_stamp: "Formatted at 2026-08-11 12:15:26 on dist-test-slave-rmhg"
server_key: "5c7bc524db519cebcf9dd7927d1a14e0"
server_key_iv: "0d66e7e988224ccf5e1b209d847da255"
server_key_version: "encryptionkey@0"
I20260811 12:15:26.852782 10521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:15:26.856455 10547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:26.858072 10521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:26.858245 10521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "bee4528070b24536ae7bc91b362a1edb"
format_stamp: "Formatted at 2026-08-11 12:15:26 on dist-test-slave-rmhg"
server_key: "5c7bc524db519cebcf9dd7927d1a14e0"
server_key_iv: "0d66e7e988224ccf5e1b209d847da255"
server_key_version: "encryptionkey@0"
I20260811 12:15:26.858460 10521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:26.877038 10521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:26.881102 10521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:26.881415 10521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:26.884369 10521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:26.884492 10521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:26.884610 10521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:26.884687 10521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:15:26 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:26.908911 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:26.884957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:42757 (local address 127.7.39.62:37675)
0811 12:15:26.885241 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:26.885247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:26.886801 (+ 1554us) server_negotiation.cc:407] Connection header received
0811 12:15:26.888937 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:26.888943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:26.889033 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:26.889123 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:26.891455 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.892488 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.894972 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.895563 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.895694 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:26.901118 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:26.901124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:26.901128 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:26.901207 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:26.904827 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:26.905910 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:26.905917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:26.905921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:26.906092 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:26.906675 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:26.906681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:26.906685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:26.907017 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:26.907194 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:26.908295 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0811 12:15:26.908544 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 12:15:26.910799 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:26.885642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37675 (local address 127.7.39.2:42757)
0811 12:15:26.886575 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:26.886659 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:26.888635 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:26.889447 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:26.889455 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:26.890204 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:26.891185 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:26.891238 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.892719 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.892724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:26.894813 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:26.894827 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.896053 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.896059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:26.896349 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:26.897176 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:26.897213 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:26.900780 (+ 3567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:26.905010 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:26.905020 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:26.905035 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:26.905691 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:26.906260 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:26.906269 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:26.906272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:26.906539 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:26.907177 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:26.907191 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:26.907758 (+ 567us) client_negotiation.cc:769] Sending connection context
0811 12:15:26.908160 (+ 402us) client_negotiation.cc:240] Negotiation successful
0811 12:15:26.908842 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":369,"threads_started":1}
I20260811 12:15:26.932102 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:26.916568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37675 (local address 127.7.39.2:49727)
0811 12:15:26.917233 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:26.917253 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:26.917393 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:26.917765 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:26.917770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:26.918372 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:26.918837 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:26.918852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.919886 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.919892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:26.921643 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:26.921665 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.922847 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.922854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:26.923070 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:26.923914 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:26.923937 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:26.924812 (+ 875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:26.928732 (+ 3920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:26.928739 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:26.928742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:26.929398 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:26.929905 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:26.929911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:26.929915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:26.930093 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:26.930821 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:26.930825 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:26.931078 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 12:15:26.931243 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 12:15:26.931601 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":336,"threads_started":1}
I20260811 12:15:26.933075 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:26.916571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:49727 (local address 127.7.39.62:37675)
0811 12:15:26.916771 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:26.916778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:26.917336 (+ 558us) server_negotiation.cc:407] Connection header received
0811 12:15:26.917519 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:26.917524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:26.917607 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:26.917714 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:26.918985 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.919756 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.921916 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:26.922465 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:26.922592 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:26.925012 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:26.925017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:26.925020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:26.925054 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:26.928577 (+ 3523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:26.929534 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:26.929540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:26.929544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:26.929728 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:26.930231 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:26.930237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:26.930240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:26.930689 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:26.930853 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:26.932478 (+ 1625us) server_negotiation.cc:299] Negotiation successful
0811 12:15:26.932753 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:15:26.933430 10521 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:35419
I20260811 12:15:26.933501 10749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:35419 every 8 connection(s)
I20260811 12:15:26.934923 10521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:26.936934 10750 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37675
I20260811 12:15:26.937179 10750 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:26.937806 10750 heartbeater.cc:511] Master 127.7.39.62:37675 requested a full tablet report, sending...
I20260811 12:15:26.939110 10194 ts_manager.cc:194] Registered new tserver with Master: bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419)
I20260811 12:15:26.939884 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:42757
I20260811 12:15:26.940306 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10521
I20260811 12:15:26.940460 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:26.940894 7324 external_mini_cluster.cc:1428] Setting key 7651ef0ef17bb6c1e5b7fdb857303eca
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:15:26.987890 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37675
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:27.322399 10754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:27.322721 10754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:27.322872 10754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:27.330359 10754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:27.330490 10754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:27.330622 10754 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:15:27.330687 10754 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:15:27.330794 10754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:27.340826 10754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:37675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:27.342280 10754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:27.343672 10754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:27.357331 10766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:27.357365 10772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:27.357565 10764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:27.358670 10754 server_base.cc:1034] running on GCE node
I20260811 12:15:27.359273 10754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:27.360212 10754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:27.361406 10754 hybrid_clock.cc:648] HybridClock initialized: now 1786450527361337 us; error 57 us; skew 500 ppm
I20260811 12:15:27.361758 10754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:27 dist-test-slave-rmhg krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450527, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:27.365187 10754 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:15:27.366772 10754 webserver.cc:460] Webserver started at http://127.7.39.3:35031/ using document root <none> and password file <none>
I20260811 12:15:27.367501 10754 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:27.367620 10754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:27.367926 10754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:27.370958 10754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "7ca842b535a949f99111414c78e83bae"
format_stamp: "Formatted at 2026-08-11 12:15:27 on dist-test-slave-rmhg"
server_key: "dbdebae7074ff0bffaebd00c826dcb86"
server_key_iv: "4ebbf53ce6a8fc6e5b09e47cbefe806b"
server_key_version: "encryptionkey@0"
I20260811 12:15:27.371744 10754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "7ca842b535a949f99111414c78e83bae"
format_stamp: "Formatted at 2026-08-11 12:15:27 on dist-test-slave-rmhg"
server_key: "dbdebae7074ff0bffaebd00c826dcb86"
server_key_iv: "4ebbf53ce6a8fc6e5b09e47cbefe806b"
server_key_version: "encryptionkey@0"
I20260811 12:15:27.377389 10754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:15:27.381175 10785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:27.382766 10754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:27.382934 10754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "7ca842b535a949f99111414c78e83bae"
format_stamp: "Formatted at 2026-08-11 12:15:27 on dist-test-slave-rmhg"
server_key: "dbdebae7074ff0bffaebd00c826dcb86"
server_key_iv: "4ebbf53ce6a8fc6e5b09e47cbefe806b"
server_key_version: "encryptionkey@0"
I20260811 12:15:27.383144 10754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:27.409965 10754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:27.414523 10754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:27.414783 10754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:27.417662 10754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:27.417795 10754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:27.418226 10516 heartbeater.cc:503] Master 127.7.39.62:37675 was elected leader, sending a full tablet report...
I20260811 12:15:27.417917 10754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:27.418715 10754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
Aug 11 12:15:27 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450527, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:27.443583 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.418215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:32941 (local address 127.7.39.62:37675)
0811 12:15:27.418387 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.418391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.420519 (+ 2128us) server_negotiation.cc:407] Connection header received
0811 12:15:27.422578 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.422585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.422675 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.422766 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:27.425054 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.426153 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.428870 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.429470 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.429594 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.434937 (+ 5343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:27.434943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:27.434946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:27.434989 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:27.438872 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:27.440325 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:27.440330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:27.440333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:27.440521 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:27.441309 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:27.441315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:27.441319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:27.441677 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:27.441798 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.442944 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.443209 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:15:27.445605 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.418882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37675 (local address 127.7.39.3:32941)
0811 12:15:27.420366 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.420405 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.422290 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.423029 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.423041 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.423824 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:27.424828 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.424864 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.426316 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.426329 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.428669 (+ 2340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.428685 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.429810 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.429821 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.430189 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.431162 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:27.431205 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:27.434640 (+ 3435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:27.439200 (+ 4560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:27.439212 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:27.439235 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:27.440094 (+ 859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:27.440719 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:27.440729 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:27.440734 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:27.441134 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:27.441824 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:27.441837 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:27.442432 (+ 595us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.442800 (+ 368us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.443508 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1186,"thread_start_us":900,"threads_started":1}
I20260811 12:15:27.467444 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.451649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:36691 (local address 127.7.39.62:37675)
0811 12:15:27.451870 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.451877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.452390 (+ 513us) server_negotiation.cc:407] Connection header received
0811 12:15:27.452568 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.452574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.452689 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.452821 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:27.454476 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.455634 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.457979 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.458463 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.458597 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.461070 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:27.461075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:27.461078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:27.461114 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:27.464289 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:27.465256 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:27.465262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:27.465265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:27.465438 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:27.465843 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:27.465849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:27.465853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:27.466258 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:27.466402 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.466848 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.467085 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:15:27.467530 10897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.451611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37675 (local address 127.7.39.3:36691)
0811 12:15:27.452277 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.452297 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.452445 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.452866 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.452871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.453752 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:27.454239 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.454257 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.455864 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.455874 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.457821 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.457839 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.458722 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.458733 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.458930 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.459639 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:27.459671 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:27.460844 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:27.464445 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:27.464453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:27.464458 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:27.465054 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:27.465581 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:27.465587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:27.465591 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:27.465729 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:27.466381 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:27.466387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:27.466571 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.466708 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.467086 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":350,"threads_started":1}
I20260811 12:15:27.471649 10754 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:45297
I20260811 12:15:27.471697 10971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:45297 every 8 connection(s)
I20260811 12:15:27.472877 10754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:27.474843 10972 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37675
I20260811 12:15:27.475033 10972 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:27.475651 10972 heartbeater.cc:511] Master 127.7.39.62:37675 requested a full tablet report, sending...
I20260811 12:15:27.476990 10194 ts_manager.cc:194] Registered new tserver with Master: 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297)
I20260811 12:15:27.477818 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:32941
I20260811 12:15:27.480234 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10754
I20260811 12:15:27.480372 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:27.480798 7324 external_mini_cluster.cc:1428] Setting key f1f490cd2d65da95d0c1fa26a847e1ac
I20260811 12:15:27.483645 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:15:27 dist-test-slave-rmhg krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:15:27 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:27.537761 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.523420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.7.39.62:37675)
0811 12:15:27.523609 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.523615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.523972 (+ 357us) server_negotiation.cc:407] Connection header received
0811 12:15:27.524141 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.524144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.524209 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.524303 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:27.525774 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.526538 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.527792 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.528231 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.528330 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.532152 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:27.532157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:27.532161 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:27.532202 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:27.534652 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:27.535482 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:27.535488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:27.535492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:27.535654 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:27.536103 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:27.536108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:27.536111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:27.536489 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:27.536733 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.537128 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.537387 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:15:27.541955 10194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:27.549002 10194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:27.584185 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.563901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.7.39.2:35419)
0811 12:15:27.576171 (+ 12270us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.576185 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.577115 (+ 930us) server_negotiation.cc:407] Connection header received
0811 12:15:27.577343 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.577352 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.577653 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.577860 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:27.578521 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.579973 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.582005 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.582960 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.583096 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.583362 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.583697 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.583882 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12034,"thread_start_us":12301,"threads_started":1}
I20260811 12:15:27.584227 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.563658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:35419 (local address 127.0.0.1:37174)
0811 12:15:27.576938 (+ 13280us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.576957 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.577136 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.577902 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.577909 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.577944 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:27.578320 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.578331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.580108 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.580114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.581812 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.581828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.583182 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.583189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.583419 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.583467 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.583615 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.583856 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13131,"thread_start_us":12971,"threads_started":1}
I20260811 12:15:27.588635 10641 tablet_service.cc:1467] Processing CreateTablet for tablet 75bede466d9b44639faf88dc3b5b704f (DEFAULT_TABLE table=test-table [id=06f41e5b0ae3430590aaff0c0e5899fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:27.590121 10641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bede466d9b44639faf88dc3b5b704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:27.598104 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.562196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.7.39.1:44545)
0811 12:15:27.591726 (+ 29530us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.591732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.591757 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:15:27.591842 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.591850 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.592070 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.592255 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:27.592894 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.594200 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.596044 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.597006 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.597143 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.597419 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.597660 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.597835 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29372,"thread_start_us":29240,"threads_started":1}
I20260811 12:15:27.598187 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.562103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44545 (local address 127.0.0.1:44158)
0811 12:15:27.563001 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.563047 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.563212 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.592284 (+ 29072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.592290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.592321 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:27.592722 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.592733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.594342 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.594352 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.595877 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.595891 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.597265 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.597272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.597461 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.597494 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.597583 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.597737 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":63,"thread_start_us":407,"threads_started":1}
I20260811 12:15:27.599433 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.564077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45297 (local address 127.0.0.1:43622)
0811 12:15:27.584505 (+ 20428us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.584526 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.584645 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.590690 (+ 6045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.590696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.590732 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:27.590931 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.590944 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.593843 (+ 2899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.593849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.595874 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.595893 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.598167 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.598177 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.598816 (+ 639us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.598850 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.598956 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.599108 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20304,"mutex_wait_us":2,"thread_start_us":26995,"threads_started":1}
I20260811 12:15:27.599488 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.564795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.7.39.3:45297)
0811 12:15:27.589182 (+ 24387us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.589192 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.589238 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:15:27.590070 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.590081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.590401 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.590696 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:27.591184 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.593676 (+ 2492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.596102 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.597647 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.597803 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.598052 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.599047 (+ 995us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.599243 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24136,"thread_start_us":24560,"threads_started":1}
I20260811 12:15:27.602396 10403 tablet_service.cc:1467] Processing CreateTablet for tablet 75bede466d9b44639faf88dc3b5b704f (DEFAULT_TABLE table=test-table [id=06f41e5b0ae3430590aaff0c0e5899fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:27.603391 10860 tablet_service.cc:1467] Processing CreateTablet for tablet 75bede466d9b44639faf88dc3b5b704f (DEFAULT_TABLE table=test-table [id=06f41e5b0ae3430590aaff0c0e5899fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:27.604046 10403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bede466d9b44639faf88dc3b5b704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:27.604954 10860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bede466d9b44639faf88dc3b5b704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:27.606993 11001 tablet_bootstrap.cc:492] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Bootstrap starting.
I20260811 12:15:27.610302 11001 tablet_bootstrap.cc:654] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:27.611331 11001 log.cc:824] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:27.614774 11001 tablet_bootstrap.cc:492] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: No bootstrap required, opened a new log
I20260811 12:15:27.615044 11001 ts_tablet_manager.cc:1397] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:15:27.620216 11004 tablet_bootstrap.cc:492] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Bootstrap starting.
I20260811 12:15:27.620774 11006 tablet_bootstrap.cc:492] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Bootstrap starting.
I20260811 12:15:27.623971 11004 tablet_bootstrap.cc:654] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:27.624385 11006 tablet_bootstrap.cc:654] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:27.624027 11001 raft_consensus.cc:348] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.624571 11001 raft_consensus.cc:374] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:27.624686 11001 raft_consensus.cc:729] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee4528070b24536ae7bc91b362a1edb, State: Initialized, Role: FOLLOWER
I20260811 12:15:27.625346 11004 log.cc:824] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:27.625468 11001 consensus_queue.cc:260] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.625721 11006 log.cc:824] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:27.630798 10750 heartbeater.cc:503] Master 127.7.39.62:37675 was elected leader, sending a full tablet report...
I20260811 12:15:27.632308 11004 tablet_bootstrap.cc:492] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: No bootstrap required, opened a new log
I20260811 12:15:27.632581 11001 ts_tablet_manager.cc:1428] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260811 12:15:27.632640 11004 ts_tablet_manager.cc:1397] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260811 12:15:27.633597 11006 tablet_bootstrap.cc:492] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: No bootstrap required, opened a new log
I20260811 12:15:27.633883 11006 ts_tablet_manager.cc:1397] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260811 12:15:27.640571 11004 raft_consensus.cc:348] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.641001 11004 raft_consensus.cc:374] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:27.641101 11004 raft_consensus.cc:729] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97463d1a263450fa63aa714db09e877, State: Initialized, Role: FOLLOWER
I20260811 12:15:27.641669 11006 raft_consensus.cc:348] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.641780 11004 consensus_queue.cc:260] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.642005 11006 raft_consensus.cc:374] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:27.642105 11006 raft_consensus.cc:729] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca842b535a949f99111414c78e83bae, State: Initialized, Role: FOLLOWER
I20260811 12:15:27.642786 11006 consensus_queue.cc:260] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.643595 11004 ts_tablet_manager.cc:1428] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 12:15:27.643977 10972 heartbeater.cc:503] Master 127.7.39.62:37675 was elected leader, sending a full tablet report...
I20260811 12:15:27.644662 11006 ts_tablet_manager.cc:1428] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
W20260811 12:15:27.663235 10517 tablet.cc:2369] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:15:27.686954 10751 tablet.cc:2369] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:15:27.724478 10973 tablet.cc:2369] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:15:27.737488 11009 raft_consensus.cc:482] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:27.737788 11009 raft_consensus.cc:504] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.739578 11009 leader_election.cc:288] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545), 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297)
I20260811 12:15:27.746731 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.740099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:54757 (local address 127.7.39.1:44545)
0811 12:15:27.740307 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.740314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.741334 (+ 1020us) server_negotiation.cc:407] Connection header received
0811 12:15:27.741510 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.741517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.741610 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.741681 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:27.742350 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.743053 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.744943 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.745786 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.745893 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.746005 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.746380 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.746502 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:15:27.746737 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.740096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:49311 (local address 127.7.39.3:45297)
0811 12:15:27.740310 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.740316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.741418 (+ 1102us) server_negotiation.cc:407] Connection header received
0811 12:15:27.741566 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.741569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.741628 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.741718 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:27.742373 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.743069 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.744939 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.745831 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.745965 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.746071 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.746347 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.746475 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:15:27.746881 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.740042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44545 (local address 127.7.39.2:54757)
0811 12:15:27.741199 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.741218 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.741364 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.741722 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.741727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.741759 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:27.742199 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.742209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.743198 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.743204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.744777 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.744790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.746034 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.746041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.746222 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.746246 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.746336 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.746467 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":9,"thread_start_us":471,"threads_started":1}
I20260811 12:15:27.747247 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.740098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45297 (local address 127.7.39.2:49311)
0811 12:15:27.741295 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.741314 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.741453 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.741751 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.741755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.741779 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:27.742217 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.742227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.743206 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.743210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.744774 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.744788 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.746066 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.746071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.746197 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.746221 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.746296 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.746426 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"mutex_wait_us":35,"thread_start_us":612,"threads_started":1}
I20260811 12:15:27.748032 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bede466d9b44639faf88dc3b5b704f" candidate_uuid: "bee4528070b24536ae7bc91b362a1edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97463d1a263450fa63aa714db09e877" is_pre_election: true
I20260811 12:15:27.748283 10884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bede466d9b44639faf88dc3b5b704f" candidate_uuid: "bee4528070b24536ae7bc91b362a1edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca842b535a949f99111414c78e83bae" is_pre_election: true
I20260811 12:15:27.748533 10435 raft_consensus.cc:2454] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bee4528070b24536ae7bc91b362a1edb in term 0.
I20260811 12:15:27.748847 10884 raft_consensus.cc:2454] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bee4528070b24536ae7bc91b362a1edb in term 0.
I20260811 12:15:27.749308 10553 leader_election.cc:302] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bee4528070b24536ae7bc91b362a1edb, d97463d1a263450fa63aa714db09e877; no voters:
I20260811 12:15:27.749760 11009 raft_consensus.cc:2781] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:27.749928 11009 raft_consensus.cc:482] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:27.750033 11009 raft_consensus.cc:3037] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:27.752259 11009 raft_consensus.cc:504] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.752936 11009 leader_election.cc:288] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [CANDIDATE]: Term 1 election: Requested vote from peers d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545), 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297)
I20260811 12:15:27.753418 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bede466d9b44639faf88dc3b5b704f" candidate_uuid: "bee4528070b24536ae7bc91b362a1edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97463d1a263450fa63aa714db09e877"
I20260811 12:15:27.753623 10435 raft_consensus.cc:3037] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:27.753721 10884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bede466d9b44639faf88dc3b5b704f" candidate_uuid: "bee4528070b24536ae7bc91b362a1edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca842b535a949f99111414c78e83bae"
I20260811 12:15:27.753949 10884 raft_consensus.cc:3037] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:27.756498 10435 raft_consensus.cc:2454] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bee4528070b24536ae7bc91b362a1edb in term 1.
I20260811 12:15:27.756592 10884 raft_consensus.cc:2454] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bee4528070b24536ae7bc91b362a1edb in term 1.
I20260811 12:15:27.756930 10553 leader_election.cc:302] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bee4528070b24536ae7bc91b362a1edb, d97463d1a263450fa63aa714db09e877; no voters:
I20260811 12:15:27.757268 11009 raft_consensus.cc:2781] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:27.757766 11009 raft_consensus.cc:686] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 1 LEADER]: Becoming Leader. State: Replica: bee4528070b24536ae7bc91b362a1edb, State: Running, Role: LEADER
I20260811 12:15:27.758334 11009 consensus_queue.cc:237] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.763410 10194 catalog_manager.cc:5495] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb reported cstate change: term changed from 0 to 1, leader changed from <none> to bee4528070b24536ae7bc91b362a1edb (127.7.39.2). New cstate: current_term: 1 leader_uuid: "bee4528070b24536ae7bc91b362a1edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:15:27 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450525, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:27.840876 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.827734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.7.39.62:37675)
0811 12:15:27.827966 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.827973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.828544 (+ 571us) server_negotiation.cc:407] Connection header received
0811 12:15:27.828679 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.828683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.828745 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.828827 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:27.830063 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.830779 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.832020 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.832439 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.832549 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.835730 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:27.835736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:27.835739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:27.835769 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:27.838195 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:27.838928 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:27.838932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:27.838935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:27.839061 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:27.839431 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:27.839435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:27.839437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:27.839721 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:27.839853 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.840274 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.840582 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":99}
I20260811 12:15:27.840907 11025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.827751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37675 (local address 127.0.0.1:45520)
0811 12:15:27.828430 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.828451 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.828584 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.828860 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.828863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.829610 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:27.829901 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.829910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.830920 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.830924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.831882 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.831894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.832621 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.832627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.832772 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.833399 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:27.833416 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:27.835544 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:27.838325 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:27.838341 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:27.838350 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:27.838810 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:27.839177 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:27.839182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:27.839185 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:27.839327 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:27.839851 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:27.839861 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:27.840017 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.840174 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.840451 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":411,"threads_started":1}
I20260811 12:15:27.846022 10194 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:45520:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:15:27.847074 10194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:27.853984 10403 tablet_service.cc:1467] Processing CreateTablet for tablet 6faede62dd7e49578d2412c496dfcf16 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49a1cb6cdac34939ab13b7193ffff081]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:15:27.854055 10641 tablet_service.cc:1467] Processing CreateTablet for tablet 6faede62dd7e49578d2412c496dfcf16 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49a1cb6cdac34939ab13b7193ffff081]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:15:27.854243 10860 tablet_service.cc:1467] Processing CreateTablet for tablet 6faede62dd7e49578d2412c496dfcf16 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49a1cb6cdac34939ab13b7193ffff081]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:15:27.854904 10403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6faede62dd7e49578d2412c496dfcf16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:27.855017 10860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6faede62dd7e49578d2412c496dfcf16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:27.855032 10641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6faede62dd7e49578d2412c496dfcf16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:27.860564 11004 tablet_bootstrap.cc:492] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: Bootstrap starting.
I20260811 12:15:27.862254 11001 tablet_bootstrap.cc:492] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: Bootstrap starting.
I20260811 12:15:27.862305 11004 tablet_bootstrap.cc:654] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:27.862691 11006 tablet_bootstrap.cc:492] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: Bootstrap starting.
I20260811 12:15:27.863881 11001 tablet_bootstrap.cc:654] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:27.864288 11006 tablet_bootstrap.cc:654] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:27.864795 11004 tablet_bootstrap.cc:492] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: No bootstrap required, opened a new log
I20260811 12:15:27.864945 11004 ts_tablet_manager.cc:1397] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 12:15:27.865774 11004 raft_consensus.cc:348] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.865991 11001 tablet_bootstrap.cc:492] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: No bootstrap required, opened a new log
I20260811 12:15:27.866034 11004 raft_consensus.cc:374] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:27.866117 11001 ts_tablet_manager.cc:1397] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 12:15:27.866145 11004 raft_consensus.cc:729] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97463d1a263450fa63aa714db09e877, State: Initialized, Role: FOLLOWER
I20260811 12:15:27.866386 11004 consensus_queue.cc:260] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.866815 11006 tablet_bootstrap.cc:492] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: No bootstrap required, opened a new log
I20260811 12:15:27.866951 11006 ts_tablet_manager.cc:1397] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 12:15:27.866830 11001 raft_consensus.cc:348] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.867018 11004 ts_tablet_manager.cc:1428] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 12:15:27.867054 11001 raft_consensus.cc:374] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:27.867141 11001 raft_consensus.cc:729] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee4528070b24536ae7bc91b362a1edb, State: Initialized, Role: FOLLOWER
I20260811 12:15:27.866920 11011 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.867317 11011 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:27.867345 11001 consensus_queue.cc:260] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.867681 11006 raft_consensus.cc:348] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.867924 11006 raft_consensus.cc:374] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:27.867926 11001 ts_tablet_manager.cc:1428] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:15:27.868010 11006 raft_consensus.cc:729] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca842b535a949f99111414c78e83bae, State: Initialized, Role: FOLLOWER
I20260811 12:15:27.867887 11009 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.868244 11006 consensus_queue.cc:260] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.868528 11009 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:27.868669 11012 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.869036 11012 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:27.869504 11006 ts_tablet_manager.cc:1428] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:15:27.886709 11012 raft_consensus.cc:482] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:27.886931 11012 raft_consensus.cc:504] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.888621 11012 leader_election.cc:288] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545), bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419)
I20260811 12:15:27.895083 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.888798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44545 (local address 127.7.39.3:34989)
0811 12:15:27.889073 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.889093 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.889237 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.889718 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.889722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.889754 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:27.889966 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.889978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.891248 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.891252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.893105 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.893121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.894477 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.894481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.894627 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.894654 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.894734 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.894830 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 12:15:27.895223 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.889135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:34989 (local address 127.7.39.1:44545)
0811 12:15:27.889344 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.889350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.889368 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:15:27.889458 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.889464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.889568 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.889667 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:27.890106 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.891114 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.893277 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.894328 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.894457 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.894564 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.894777 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.894904 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 12:15:27.895888 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.889213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:50613 (local address 127.7.39.2:35419)
0811 12:15:27.889398 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.889404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.889753 (+ 349us) server_negotiation.cc:407] Connection header received
0811 12:15:27.890082 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.890087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.890163 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.890256 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:27.890743 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.891622 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.893723 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.894876 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.895009 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.895112 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.895519 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.895653 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 12:15:27.895905 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.888868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:35419 (local address 127.7.39.3:50613)
0811 12:15:27.889650 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.889668 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.889913 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.890285 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.890289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.890309 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:27.890578 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.890586 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.891754 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.891759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.893564 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.893575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.895115 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.895121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.895304 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.895338 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.895461 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.895554 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":1,"spinlock_wait_cycles":24960,"thread_start_us":513,"threads_started":1}
I20260811 12:15:27.895869 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6faede62dd7e49578d2412c496dfcf16" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97463d1a263450fa63aa714db09e877" is_pre_election: true
I20260811 12:15:27.896144 10435 raft_consensus.cc:2454] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 0.
I20260811 12:15:27.896646 10788 leader_election.cc:302] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca842b535a949f99111414c78e83bae, d97463d1a263450fa63aa714db09e877; no voters:
I20260811 12:15:27.896822 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6faede62dd7e49578d2412c496dfcf16" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bee4528070b24536ae7bc91b362a1edb" is_pre_election: true
I20260811 12:15:27.897135 11012 raft_consensus.cc:2781] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:27.897285 11012 raft_consensus.cc:482] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:27.897373 11012 raft_consensus.cc:3037] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:27.897289 10675 raft_consensus.cc:2454] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 0.
I20260811 12:15:27.899859 11012 raft_consensus.cc:504] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.900395 11012 leader_election.cc:288] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 election: Requested vote from peers d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545), bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419)
I20260811 12:15:27.900928 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6faede62dd7e49578d2412c496dfcf16" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97463d1a263450fa63aa714db09e877"
I20260811 12:15:27.900939 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6faede62dd7e49578d2412c496dfcf16" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bee4528070b24536ae7bc91b362a1edb"
I20260811 12:15:27.901170 10675 raft_consensus.cc:3037] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:27.901189 10435 raft_consensus.cc:3037] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:27.904168 10675 raft_consensus.cc:2454] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 1.
I20260811 12:15:27.904151 10435 raft_consensus.cc:2454] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 1.
I20260811 12:15:27.904644 10788 leader_election.cc:302] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7ca842b535a949f99111414c78e83bae, bee4528070b24536ae7bc91b362a1edb, d97463d1a263450fa63aa714db09e877; no voters:
I20260811 12:15:27.904986 11012 raft_consensus.cc:2781] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:27.905473 11012 raft_consensus.cc:686] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [term 1 LEADER]: Becoming Leader. State: Replica: 7ca842b535a949f99111414c78e83bae, State: Running, Role: LEADER
I20260811 12:15:27.906006 11012 consensus_queue.cc:237] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:27.910367 11032 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: TxnStatusTablet state changed. Reason: New leader 7ca842b535a949f99111414c78e83bae. Latest consensus state: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.910763 11032 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:15:27.911085 10194 catalog_manager.cc:5495] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ca842b535a949f99111414c78e83bae (127.7.39.3). New cstate: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } health_report { overall_health: HEALTHY } } }
I20260811 12:15:27.911433 11036 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:15:27.911716 11036 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:15:27.936303 11025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.930511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45297 (local address 127.0.0.1:43628)
0811 12:15:27.930795 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:27.930816 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:27.930964 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:27.931508 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:27.931513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:27.931527 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:15:27.931741 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.931754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.932935 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.932940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.933981 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:27.933994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.934690 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.934696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:27.934861 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.934922 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:15:27.935734 (+ 812us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:15:27.935759 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:15:27.935839 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:15:27.936003 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:15:27.936297 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.930535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43628 (local address 127.7.39.3:45297)
0811 12:15:27.930859 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.930864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.930882 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:15:27.931212 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.931217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.931328 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.931469 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:27.931877 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.932727 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.934122 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.934539 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.934659 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.935100 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:27.935625 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:27.935763 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.935869 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.935965 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260811 12:15:27.944559 10675 raft_consensus.cc:1261] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Refusing update from remote peer 7ca842b535a949f99111414c78e83bae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:27.944787 10435 raft_consensus.cc:1261] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Refusing update from remote peer 7ca842b535a949f99111414c78e83bae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:27.945586 11032 consensus_queue.cc:1035] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:27.946004 11012 consensus_queue.cc:1035] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae [LEADER]: Connected to new peer: Peer: permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:27.962488 11009 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: TxnStatusTablet state changed. Reason: New leader 7ca842b535a949f99111414c78e83bae. Latest consensus state: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.962872 11009 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:27.968497 11011 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: TxnStatusTablet state changed. Reason: New leader 7ca842b535a949f99111414c78e83bae. Latest consensus state: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.968829 11011 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:27.976586 11043 mvcc.cc:205] Tried to move back new op lower bound from 7317301362451353600 to 7317301362307133440. Current Snapshot: MvccSnapshot[applied={T|T < 7317301362451353600}]
I20260811 12:15:27.978554 11009 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.978909 11009 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P bee4528070b24536ae7bc91b362a1edb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:27.981011 11011 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.981292 11011 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P d97463d1a263450fa63aa714db09e877: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:15:27.982169 11012 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.982432 11012 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:15:27.983433 11032 tablet_replica.cc:439] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } } }
I20260811 12:15:27.983743 11032 tablet_replica.cc:442] T 6faede62dd7e49578d2412c496dfcf16 P 7ca842b535a949f99111414c78e83bae: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:15:27.994683 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:27.988358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.7.39.2:35419)
0811 12:15:27.988543 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:27.988549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:27.988649 (+ 100us) server_negotiation.cc:407] Connection header received
0811 12:15:27.988835 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:27.988840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:27.988980 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:27.989241 (+ 261us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:27.989548 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.990688 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.992097 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:27.992733 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:27.992861 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:27.993309 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:27.994092 (+ 783us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:27.994240 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:27.994286 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 12:15:27.994376 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:15:28.013798 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:28.007223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:41281 (local address 127.7.39.3:45297)
0811 12:15:28.007451 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:28.007458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:28.008217 (+ 759us) server_negotiation.cc:407] Connection header received
0811 12:15:28.008406 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:28.008412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:28.008508 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:28.008614 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:28.009183 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.010333 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.011734 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.012434 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.012574 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:28.013028 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:28.013212 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:28.013307 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:28.013412 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 12:15:28.013499 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:15:28.013823 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:28.007195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45297 (local address 127.7.39.2:41281)
0811 12:15:28.008094 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:28.008111 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:28.008267 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:28.008617 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:28.008620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:28.008629 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:15:28.009000 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:28.009013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.010471 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.010475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:28.011578 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:28.011590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.012605 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.012609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:28.012744 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:28.012807 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:15:28.013302 (+ 495us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:15:28.013311 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:15:28.013377 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 12:15:28.013498 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":464,"threads_started":1}
I20260811 12:15:28.023523 10435 raft_consensus.cc:1261] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Refusing update from remote peer bee4528070b24536ae7bc91b362a1edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:28.023790 10884 raft_consensus.cc:1261] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Refusing update from remote peer bee4528070b24536ae7bc91b362a1edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:28.024266 11015 consensus_queue.cc:1035] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:28.025138 11015 consensus_queue.cc:1035] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:28.033032 11041 mvcc.cc:205] Tried to move back new op lower bound from 7317301362777681920 to 7317301361702240256. Current Snapshot: MvccSnapshot[applied={T|T < 7317301362777681920}]
I20260811 12:15:28.039779 11040 mvcc.cc:205] Tried to move back new op lower bound from 7317301362777681920 to 7317301361702240256. Current Snapshot: MvccSnapshot[applied={T|T < 7317301362777681920}]
I20260811 12:15:28.073947 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:28.066949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:42601 (local address 127.7.39.2:35419)
0811 12:15:28.067176 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:28.067183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:28.067805 (+ 622us) server_negotiation.cc:407] Connection header received
0811 12:15:28.068001 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:28.068004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:28.068073 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:28.068146 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:28.068737 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.069779 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.072191 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.072624 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.072741 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:28.073334 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:28.073445 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:28.073516 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:28.073652 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 12:15:28.073736 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:15:28.074252 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:28.066944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:35419 (local address 127.7.39.3:42601)
0811 12:15:28.067694 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:28.067715 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:28.067868 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:28.068188 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:28.068193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:28.068210 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:15:28.068580 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:28.068593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.070135 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.070140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:28.072013 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:28.072031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.072827 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.072834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:28.073012 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:28.073102 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:15:28.073541 (+ 439us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:15:28.073553 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:15:28.073638 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 12:15:28.073800 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":360,"threads_started":1}
I20260811 12:15:28.110906 10192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:28.111235 10192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:28.115440 10192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: Sending DeleteTablet for 3 replicas of tablet 75bede466d9b44639faf88dc3b5b704f
I20260811 12:15:28.116660 10641 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bede466d9b44639faf88dc3b5b704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:28 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:37174
I20260811 12:15:28.116665 10403 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bede466d9b44639faf88dc3b5b704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:28 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:44158
I20260811 12:15:28.119320 10194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:28.120029 10194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:28.128736 10640 tablet_service.cc:1467] Processing CreateTablet for tablet a289160b41e145a3b8348074ce75ceb6 (DEFAULT_TABLE table=test-table [id=9e7091bc07354ec89a662530f82bc819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:28.128734 10402 tablet_service.cc:1467] Processing CreateTablet for tablet a289160b41e145a3b8348074ce75ceb6 (DEFAULT_TABLE table=test-table [id=9e7091bc07354ec89a662530f82bc819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:28.129360 10402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a289160b41e145a3b8348074ce75ceb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:28.129360 10640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a289160b41e145a3b8348074ce75ceb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:28.133106 11064 tablet_replica.cc:330] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: stopping tablet replica
I20260811 12:15:28.133607 11064 raft_consensus.cc:2229] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:15:28.134421 11064 raft_consensus.cc:2258] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:28.135727 11004 tablet_bootstrap.cc:492] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: Bootstrap starting.
I20260811 12:15:28.136251 10859 tablet_service.cc:1467] Processing CreateTablet for tablet a289160b41e145a3b8348074ce75ceb6 (DEFAULT_TABLE table=test-table [id=9e7091bc07354ec89a662530f82bc819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:28.136808 10859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a289160b41e145a3b8348074ce75ceb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:28.137483 11004 tablet_bootstrap.cc:654] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:28.136341 10860 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bede466d9b44639faf88dc3b5b704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:28 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:43622
I20260811 12:15:28.142647 11064 ts_tablet_manager.cc:1905] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:28.143869 11006 tablet_bootstrap.cc:492] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: Bootstrap starting.
I20260811 12:15:28.145727 11006 tablet_bootstrap.cc:654] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:28.150923 11064 ts_tablet_manager.cc:1918] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:28.151149 11064 log.cc:1196] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/75bede466d9b44639faf88dc3b5b704f
I20260811 12:15:28.151763 11064 ts_tablet_manager.cc:1939] T 75bede466d9b44639faf88dc3b5b704f P bee4528070b24536ae7bc91b362a1edb: Deleting consensus metadata
I20260811 12:15:28.152908 11001 tablet_bootstrap.cc:492] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: Bootstrap starting.
I20260811 12:15:28.154110 11004 tablet_bootstrap.cc:492] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: No bootstrap required, opened a new log
I20260811 12:15:28.154222 10173 catalog_manager.cc:4841] TS bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419): tablet 75bede466d9b44639faf88dc3b5b704f (table test-table [id=06f41e5b0ae3430590aaff0c0e5899fa]) successfully deleted
I20260811 12:15:28.154296 11004 ts_tablet_manager.cc:1397] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260811 12:15:28.154742 11001 tablet_bootstrap.cc:654] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:28.155072 11004 raft_consensus.cc:348] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.155328 11004 raft_consensus.cc:374] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:28.155392 11004 raft_consensus.cc:729] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97463d1a263450fa63aa714db09e877, State: Initialized, Role: FOLLOWER
I20260811 12:15:28.155632 11004 consensus_queue.cc:260] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.156198 11004 ts_tablet_manager.cc:1428] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 12:15:28.162840 11069 tablet_replica.cc:330] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: stopping tablet replica
I20260811 12:15:28.163038 11006 tablet_bootstrap.cc:492] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: No bootstrap required, opened a new log
I20260811 12:15:28.163266 11006 ts_tablet_manager.cc:1397] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.005s
I20260811 12:15:28.163637 11069 raft_consensus.cc:2229] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:28.164106 11069 raft_consensus.cc:2258] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:28.164870 11001 tablet_bootstrap.cc:492] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: No bootstrap required, opened a new log
I20260811 12:15:28.165141 11001 ts_tablet_manager.cc:1397] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260811 12:15:28.166143 11001 raft_consensus.cc:348] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.166501 11001 raft_consensus.cc:374] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:28.166644 11001 raft_consensus.cc:729] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee4528070b24536ae7bc91b362a1edb, State: Initialized, Role: FOLLOWER
I20260811 12:15:28.167012 11001 consensus_queue.cc:260] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.167199 11069 ts_tablet_manager.cc:1905] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:28.169588 11006 raft_consensus.cc:348] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.169848 11006 raft_consensus.cc:374] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:28.169938 11006 raft_consensus.cc:729] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca842b535a949f99111414c78e83bae, State: Initialized, Role: FOLLOWER
I20260811 12:15:28.170217 11006 consensus_queue.cc:260] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.171448 11006 ts_tablet_manager.cc:1428] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 12:15:28.171733 11001 ts_tablet_manager.cc:1428] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 12:15:28.173804 11066 tablet_replica.cc:330] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: stopping tablet replica
I20260811 12:15:28.174319 11066 raft_consensus.cc:2229] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:28.174750 11066 raft_consensus.cc:2258] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:28.177240 11066 ts_tablet_manager.cc:1905] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:28.183426 11066 ts_tablet_manager.cc:1918] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:28.183616 11066 log.cc:1196] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/75bede466d9b44639faf88dc3b5b704f
I20260811 12:15:28.184165 11066 ts_tablet_manager.cc:1939] T 75bede466d9b44639faf88dc3b5b704f P d97463d1a263450fa63aa714db09e877: Deleting consensus metadata
I20260811 12:15:28.186411 10171 catalog_manager.cc:4841] TS d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545): tablet 75bede466d9b44639faf88dc3b5b704f (table test-table [id=06f41e5b0ae3430590aaff0c0e5899fa]) successfully deleted
I20260811 12:15:28.189646 11069 ts_tablet_manager.cc:1918] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:28.189841 11069 log.cc:1196] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/75bede466d9b44639faf88dc3b5b704f
I20260811 12:15:28.190709 11069 ts_tablet_manager.cc:1939] T 75bede466d9b44639faf88dc3b5b704f P 7ca842b535a949f99111414c78e83bae: Deleting consensus metadata
I20260811 12:15:28.192492 10170 catalog_manager.cc:4841] TS 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297): tablet 75bede466d9b44639faf88dc3b5b704f (table test-table [id=06f41e5b0ae3430590aaff0c0e5899fa]) successfully deleted
I20260811 12:15:28.303339 11032 raft_consensus.cc:482] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:28.303628 11032 raft_consensus.cc:504] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.304576 11032 leader_election.cc:288] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545), bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419)
I20260811 12:15:28.305269 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a289160b41e145a3b8348074ce75ceb6" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97463d1a263450fa63aa714db09e877" is_pre_election: true
I20260811 12:15:28.305557 10435 raft_consensus.cc:2454] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 0.
I20260811 12:15:28.306339 10788 leader_election.cc:302] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca842b535a949f99111414c78e83bae, d97463d1a263450fa63aa714db09e877; no voters:
I20260811 12:15:28.306639 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a289160b41e145a3b8348074ce75ceb6" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bee4528070b24536ae7bc91b362a1edb" is_pre_election: true
I20260811 12:15:28.306906 11032 raft_consensus.cc:2781] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:28.307013 11032 raft_consensus.cc:482] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:28.306998 10675 raft_consensus.cc:2454] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 0.
I20260811 12:15:28.307116 11032 raft_consensus.cc:3037] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:28.310932 11032 raft_consensus.cc:504] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.312114 11032 leader_election.cc:288] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 election: Requested vote from peers d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545), bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419)
I20260811 12:15:28.312908 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a289160b41e145a3b8348074ce75ceb6" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97463d1a263450fa63aa714db09e877"
I20260811 12:15:28.312924 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a289160b41e145a3b8348074ce75ceb6" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bee4528070b24536ae7bc91b362a1edb"
I20260811 12:15:28.313202 10435 raft_consensus.cc:3037] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:28.313220 10675 raft_consensus.cc:3037] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:28.316475 10435 raft_consensus.cc:2454] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 1.
I20260811 12:15:28.316566 10675 raft_consensus.cc:2454] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 1.
I20260811 12:15:28.317188 10788 leader_election.cc:302] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca842b535a949f99111414c78e83bae, d97463d1a263450fa63aa714db09e877; no voters:
I20260811 12:15:28.317776 11032 raft_consensus.cc:2781] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:28.318068 11032 raft_consensus.cc:686] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 1 LEADER]: Becoming Leader. State: Replica: 7ca842b535a949f99111414c78e83bae, State: Running, Role: LEADER
I20260811 12:15:28.318385 11032 consensus_queue.cc:237] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:28.321866 10194 catalog_manager.cc:5495] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ca842b535a949f99111414c78e83bae (127.7.39.3). New cstate: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } health_report { overall_health: HEALTHY } } }
I20260811 12:15:28.366391 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:28.357224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.7.39.3:45297)
0811 12:15:28.357452 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:28.357458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:28.357478 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:15:28.357546 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:28.357555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:28.357646 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:28.357737 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:28.358379 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.359639 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.362444 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.363706 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.363845 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:28.364816 (+ 971us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:28.365078 (+ 262us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:28.365181 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:28.365742 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 12:15:28.365833 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:15:28.390341 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:28.381311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:48551 (local address 127.7.39.3:45297)
0811 12:15:28.381681 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:28.381716 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:28.383029 (+ 1313us) server_negotiation.cc:407] Connection header received
0811 12:15:28.383152 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:28.383157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:28.383246 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:28.383346 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:28.383832 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.385449 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.387711 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.388600 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.388744 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:28.389313 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:28.389464 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:28.389589 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:28.389747 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 12:15:28.389847 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":126}
I20260811 12:15:28.390364 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:28.382591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45297 (local address 127.7.39.3:48551)
0811 12:15:28.382841 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:28.382863 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:28.383026 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:28.383380 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:28.383385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:28.383398 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:15:28.383664 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:28.383677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.385728 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.385734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:28.387536 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:28.387566 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:28.388793 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:28.388799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:28.388990 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:28.389047 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:15:28.389604 (+ 557us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:15:28.389617 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 12:15:28.389712 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:15:28.389871 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 12:15:28.401696 10675 raft_consensus.cc:1261] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Refusing update from remote peer 7ca842b535a949f99111414c78e83bae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:28.401882 10435 raft_consensus.cc:1261] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Refusing update from remote peer 7ca842b535a949f99111414c78e83bae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:28.402498 11032 consensus_queue.cc:1035] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [LEADER]: Connected to new peer: Peer: permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:28.403113 11076 consensus_queue.cc:1035] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:15:28.476598 11040 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 12:15:30.122385 10194 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:30.122679 10194 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:30.128038 10194 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: Sending DeleteTablet for 3 replicas of tablet a289160b41e145a3b8348074ce75ceb6
I20260811 12:15:30.129051 10860 tablet_service.cc:1514] Processing DeleteTablet for tablet a289160b41e145a3b8348074ce75ceb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:30 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:43622
I20260811 12:15:30.129114 10403 tablet_service.cc:1514] Processing DeleteTablet for tablet a289160b41e145a3b8348074ce75ceb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:30 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:44158
I20260811 12:15:30.129405 10640 tablet_service.cc:1514] Processing DeleteTablet for tablet a289160b41e145a3b8348074ce75ceb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:30 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:37174
I20260811 12:15:30.141295 11122 tablet_replica.cc:330] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: stopping tablet replica
I20260811 12:15:30.141939 11122 raft_consensus.cc:2229] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:30.142717 11122 raft_consensus.cc:2258] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:30.144178 11122 ts_tablet_manager.cc:1905] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:30.144906 11124 tablet_replica.cc:330] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: stopping tablet replica
I20260811 12:15:30.145594 11124 raft_consensus.cc:2229] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:30.146104 11124 raft_consensus.cc:2258] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:30.147199 11121 tablet_replica.cc:330] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: stopping tablet replica
I20260811 12:15:30.147425 11124 ts_tablet_manager.cc:1905] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:30.147482 11121 raft_consensus.cc:2229] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:15:30.148510 11121 raft_consensus.cc:2258] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:30.149816 11121 ts_tablet_manager.cc:1905] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:30.152774 11122 ts_tablet_manager.cc:1918] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:15:30.152971 11122 log.cc:1196] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/a289160b41e145a3b8348074ce75ceb6
I20260811 12:15:30.153563 11122 ts_tablet_manager.cc:1939] T a289160b41e145a3b8348074ce75ceb6 P d97463d1a263450fa63aa714db09e877: Deleting consensus metadata
I20260811 12:15:30.154969 10171 catalog_manager.cc:4841] TS d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545): tablet a289160b41e145a3b8348074ce75ceb6 (table test-table [id=9e7091bc07354ec89a662530f82bc819]) successfully deleted
I20260811 12:15:30.156376 11124 ts_tablet_manager.cc:1918] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:15:30.156648 11124 log.cc:1196] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/a289160b41e145a3b8348074ce75ceb6
I20260811 12:15:30.157289 11124 ts_tablet_manager.cc:1939] T a289160b41e145a3b8348074ce75ceb6 P bee4528070b24536ae7bc91b362a1edb: Deleting consensus metadata
I20260811 12:15:30.158020 11121 ts_tablet_manager.cc:1918] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:15:30.158219 11121 log.cc:1196] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/a289160b41e145a3b8348074ce75ceb6
I20260811 12:15:30.158913 11121 ts_tablet_manager.cc:1939] T a289160b41e145a3b8348074ce75ceb6 P 7ca842b535a949f99111414c78e83bae: Deleting consensus metadata
I20260811 12:15:30.159147 10173 catalog_manager.cc:4841] TS bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419): tablet a289160b41e145a3b8348074ce75ceb6 (table test-table [id=9e7091bc07354ec89a662530f82bc819]) successfully deleted
I20260811 12:15:30.159965 10170 catalog_manager.cc:4841] TS 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297): tablet a289160b41e145a3b8348074ce75ceb6 (table test-table [id=9e7091bc07354ec89a662530f82bc819]) successfully deleted
Aug 11 12:15:30 dist-test-slave-rmhg krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450530, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:15:30 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450530, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:30.204604 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.187655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.7.39.62:37675)
0811 12:15:30.188633 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:30.188637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:30.188659 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:15:30.188716 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:30.188720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:30.188781 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:30.188880 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:30.190353 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.191622 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.192994 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.193840 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.193974 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.198191 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:30.198198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:30.198202 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:30.198275 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:30.201985 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:30.202703 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:30.202707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:30.202709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:30.202987 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:30.203297 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:30.203300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:30.203302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:30.203575 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:30.203805 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:30.204054 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 12:15:30.204259 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":658,"threads_started":1}
I20260811 12:15:30.208474 10194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:30.209208 10194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:30.216208 10640 tablet_service.cc:1467] Processing CreateTablet for tablet b6f7835dcbc5493ba6c445425890b23a (DEFAULT_TABLE table=test-table [id=e541e4a81d894f618da2009c64aa9960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:30.216250 10403 tablet_service.cc:1467] Processing CreateTablet for tablet b6f7835dcbc5493ba6c445425890b23a (DEFAULT_TABLE table=test-table [id=e541e4a81d894f618da2009c64aa9960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:30.216604 10860 tablet_service.cc:1467] Processing CreateTablet for tablet b6f7835dcbc5493ba6c445425890b23a (DEFAULT_TABLE table=test-table [id=e541e4a81d894f618da2009c64aa9960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:30.216779 10640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f7835dcbc5493ba6c445425890b23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:30.216816 10403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f7835dcbc5493ba6c445425890b23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:30.217023 10860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f7835dcbc5493ba6c445425890b23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:30.225437 11136 tablet_bootstrap.cc:492] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: Bootstrap starting.
I20260811 12:15:30.228118 11136 tablet_bootstrap.cc:654] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:30.232085 11137 tablet_bootstrap.cc:492] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: Bootstrap starting.
I20260811 12:15:30.232400 11138 tablet_bootstrap.cc:492] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: Bootstrap starting.
I20260811 12:15:30.235575 11137 tablet_bootstrap.cc:654] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:30.235648 11138 tablet_bootstrap.cc:654] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:30.235538 11136 tablet_bootstrap.cc:492] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: No bootstrap required, opened a new log
I20260811 12:15:30.236184 11136 ts_tablet_manager.cc:1397] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260811 12:15:30.237320 11136 raft_consensus.cc:348] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.237705 11136 raft_consensus.cc:374] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:30.237808 11136 raft_consensus.cc:729] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97463d1a263450fa63aa714db09e877, State: Initialized, Role: FOLLOWER
I20260811 12:15:30.238170 11136 consensus_queue.cc:260] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.239744 11137 tablet_bootstrap.cc:492] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: No bootstrap required, opened a new log
I20260811 12:15:30.239985 11137 ts_tablet_manager.cc:1397] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 12:15:30.240301 11138 tablet_bootstrap.cc:492] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: No bootstrap required, opened a new log
I20260811 12:15:30.240373 11136 ts_tablet_manager.cc:1428] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 12:15:30.240562 11138 ts_tablet_manager.cc:1397] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260811 12:15:30.241230 11137 raft_consensus.cc:348] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.241539 11137 raft_consensus.cc:374] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:30.241658 11137 raft_consensus.cc:729] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee4528070b24536ae7bc91b362a1edb, State: Initialized, Role: FOLLOWER
I20260811 12:15:30.241452 11138 raft_consensus.cc:348] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.241770 11138 raft_consensus.cc:374] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:30.241997 11137 consensus_queue.cc:260] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.242216 11138 raft_consensus.cc:729] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca842b535a949f99111414c78e83bae, State: Initialized, Role: FOLLOWER
I20260811 12:15:30.242596 11138 consensus_queue.cc:260] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.243321 11138 ts_tablet_manager.cc:1428] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 12:15:30.250276 11137 ts_tablet_manager.cc:1428] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: Time spent starting tablet: real 0.010s user 0.000s sys 0.010s
I20260811 12:15:30.331449 11146 raft_consensus.cc:482] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:30.331918 11146 raft_consensus.cc:504] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.333920 11146 leader_election.cc:288] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419), 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297)
I20260811 12:15:30.356928 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.334326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45297 (local address 127.7.39.1:32849)
0811 12:15:30.346833 (+ 12507us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:30.346860 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:30.347048 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:30.348679 (+ 1631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:30.348685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:30.348714 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:30.349420 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:30.349442 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.351247 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.351253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:30.353878 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:30.353893 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.355983 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.355992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:30.356173 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.356211 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:15:30.356312 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 12:15:30.356469 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12309,"mutex_wait_us":3,"thread_start_us":12828,"threads_started":1}
I20260811 12:15:30.357002 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.334578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:32849 (local address 127.7.39.3:45297)
0811 12:15:30.348341 (+ 13763us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:30.348346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:30.348389 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:15:30.348453 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:30.348457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:30.348514 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:30.348604 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:30.349670 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.351061 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.354117 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.355688 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.355790 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.355963 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:30.356350 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 12:15:30.356464 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13549,"thread_start_us":13272,"threads_started":1}
I20260811 12:15:30.358407 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.334348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:35419 (local address 127.7.39.1:39583)
0811 12:15:30.347537 (+ 13189us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:30.347561 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:30.347747 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:30.349167 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:30.349177 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:30.349205 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:30.350332 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:30.350349 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.352266 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.352270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:30.355073 (+ 2803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:30.355090 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.357414 (+ 2324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.357425 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:30.357623 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.357668 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 12:15:30.357742 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 12:15:30.357868 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12961,"thread_start_us":11906,"threads_started":1}
I20260811 12:15:30.358543 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.334350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:39583 (local address 127.7.39.2:35419)
0811 12:15:30.348776 (+ 14426us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:30.348782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:30.348836 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:15:30.348912 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:30.348917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:30.348992 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:30.349115 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:30.350595 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.352106 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.355259 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.356799 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.356943 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.357111 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:30.357808 (+ 697us) server_negotiation.cc:299] Negotiation successful
0811 12:15:30.357998 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14234,"thread_start_us":14067,"threads_started":1}
I20260811 12:15:30.358534 10884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f7835dcbc5493ba6c445425890b23a" candidate_uuid: "d97463d1a263450fa63aa714db09e877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca842b535a949f99111414c78e83bae" is_pre_election: true
I20260811 12:15:30.358814 10884 raft_consensus.cc:2454] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d97463d1a263450fa63aa714db09e877 in term 0.
I20260811 12:15:30.359513 10321 leader_election.cc:302] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca842b535a949f99111414c78e83bae, d97463d1a263450fa63aa714db09e877; no voters:
I20260811 12:15:30.359622 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f7835dcbc5493ba6c445425890b23a" candidate_uuid: "d97463d1a263450fa63aa714db09e877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bee4528070b24536ae7bc91b362a1edb" is_pre_election: true
I20260811 12:15:30.359911 10675 raft_consensus.cc:2454] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d97463d1a263450fa63aa714db09e877 in term 0.
I20260811 12:15:30.360080 11146 raft_consensus.cc:2781] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:30.360219 11146 raft_consensus.cc:482] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:30.360338 11146 raft_consensus.cc:3037] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:30.363649 11146 raft_consensus.cc:504] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.364723 11146 leader_election.cc:288] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [CANDIDATE]: Term 1 election: Requested vote from peers bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419), 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297)
I20260811 12:15:30.365044 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f7835dcbc5493ba6c445425890b23a" candidate_uuid: "d97463d1a263450fa63aa714db09e877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bee4528070b24536ae7bc91b362a1edb"
I20260811 12:15:30.365339 10675 raft_consensus.cc:3037] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:30.365429 10884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f7835dcbc5493ba6c445425890b23a" candidate_uuid: "d97463d1a263450fa63aa714db09e877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca842b535a949f99111414c78e83bae"
I20260811 12:15:30.365823 10884 raft_consensus.cc:3037] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:30.368436 10675 raft_consensus.cc:2454] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d97463d1a263450fa63aa714db09e877 in term 1.
I20260811 12:15:30.368714 10884 raft_consensus.cc:2454] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d97463d1a263450fa63aa714db09e877 in term 1.
I20260811 12:15:30.368948 10324 leader_election.cc:302] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bee4528070b24536ae7bc91b362a1edb, d97463d1a263450fa63aa714db09e877; no voters:
I20260811 12:15:30.369483 11146 raft_consensus.cc:2781] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:30.370400 11146 raft_consensus.cc:686] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 1 LEADER]: Becoming Leader. State: Replica: d97463d1a263450fa63aa714db09e877, State: Running, Role: LEADER
I20260811 12:15:30.371165 11146 consensus_queue.cc:237] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.375270 10192 catalog_manager.cc:5495] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 reported cstate change: term changed from 0 to 1, leader changed from <none> to d97463d1a263450fa63aa714db09e877 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "d97463d1a263450fa63aa714db09e877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } health_report { overall_health: UNKNOWN } } }
I20260811 12:15:30.438645 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.430661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.7.39.1:44545)
0811 12:15:30.431575 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:30.431580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:30.431606 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:15:30.431715 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:30.431719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:30.431799 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:30.431962 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:30.432729 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.433911 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.435494 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.436246 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.436369 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.436935 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:30.437687 (+ 752us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:30.437835 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:30.438080 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 12:15:30.438182 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":466,"threads_started":1}
I20260811 12:15:30.460932 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.452392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:60199 (local address 127.7.39.3:45297)
0811 12:15:30.452624 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:30.452629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:30.453781 (+ 1152us) server_negotiation.cc:407] Connection header received
0811 12:15:30.453946 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:30.453951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:30.454046 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:30.454131 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:30.454989 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.456016 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.458381 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.459304 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.459430 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.460068 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:30.460225 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:30.460315 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:30.460599 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 12:15:30.460684 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:15:30.461392 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.452275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45297 (local address 127.7.39.1:60199)
0811 12:15:30.453651 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:30.453677 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:30.453817 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:30.454149 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:30.454153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:30.454162 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:15:30.454792 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:30.454806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.456205 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.456210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:30.458198 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:30.458216 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.459592 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.459600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:30.459760 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.459830 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:15:30.460409 (+ 579us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:15:30.460428 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 12:15:30.460528 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 12:15:30.460741 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1187,"thread_start_us":903,"threads_started":1}
I20260811 12:15:30.471095 10884 raft_consensus.cc:1261] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Refusing update from remote peer d97463d1a263450fa63aa714db09e877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:30.471422 10675 raft_consensus.cc:1261] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Refusing update from remote peer d97463d1a263450fa63aa714db09e877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:30.472018 11155 consensus_queue.cc:1035] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 12:15:30.472710 11146 consensus_queue.cc:1035] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:30.521562 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.511448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:58261 (local address 127.7.39.1:44545)
0811 12:15:30.511655 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:30.511662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:30.513662 (+ 2000us) server_negotiation.cc:407] Connection header received
0811 12:15:30.513949 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:30.513955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:30.514026 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:30.514118 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:30.515241 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.516758 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.519438 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.520297 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.520405 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.520919 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:30.521034 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:30.521122 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:30.521238 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 12:15:30.521312 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:15:30.521878 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.511646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44545 (local address 127.7.39.3:58261)
0811 12:15:30.513561 (+ 1915us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:30.513581 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:30.513788 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:30.514396 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:30.514400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:30.514413 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:15:30.515072 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:30.515086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.516996 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.517000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:30.519261 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:30.519277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.520479 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.520484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:30.520684 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.520727 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:15:30.521121 (+ 394us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:15:30.521129 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:15:30.521207 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 12:15:30.521324 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1732,"thread_start_us":1371,"threads_started":1}
I20260811 12:15:30.566371 10192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:30.566713 10192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:30.570536 10192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: Sending DeleteTablet for 3 replicas of tablet b6f7835dcbc5493ba6c445425890b23a
I20260811 12:15:30.571631 10403 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f7835dcbc5493ba6c445425890b23a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:30 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:44158
I20260811 12:15:30.571655 10640 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f7835dcbc5493ba6c445425890b23a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:30 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:37174
I20260811 12:15:30.571868 10860 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f7835dcbc5493ba6c445425890b23a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:30 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:43622
I20260811 12:15:30.571976 11124 tablet_replica.cc:330] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: stopping tablet replica
I20260811 12:15:30.572094 11122 tablet_replica.cc:330] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: stopping tablet replica
I20260811 12:15:30.572202 11124 raft_consensus.cc:2229] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:30.572243 11121 tablet_replica.cc:330] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: stopping tablet replica
I20260811 12:15:30.572422 11124 raft_consensus.cc:2258] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:30.572547 11122 raft_consensus.cc:2229] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:15:30.572569 11121 raft_consensus.cc:2229] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:30.572904 11121 raft_consensus.cc:2258] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:30.573266 11122 raft_consensus.cc:2258] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:30.574256 11124 ts_tablet_manager.cc:1905] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:30.574644 11122 ts_tablet_manager.cc:1905] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:30.574975 11121 ts_tablet_manager.cc:1905] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:30.580896 10192 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:30.581709 10192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:30.582015 11124 ts_tablet_manager.cc:1918] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:30.582190 11124 log.cc:1196] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/b6f7835dcbc5493ba6c445425890b23a
I20260811 12:15:30.582674 11124 ts_tablet_manager.cc:1939] T b6f7835dcbc5493ba6c445425890b23a P bee4528070b24536ae7bc91b362a1edb: Deleting consensus metadata
I20260811 12:15:30.583541 11122 ts_tablet_manager.cc:1918] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:30.583804 11122 log.cc:1196] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/b6f7835dcbc5493ba6c445425890b23a
I20260811 12:15:30.583875 10173 catalog_manager.cc:4841] TS bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419): tablet b6f7835dcbc5493ba6c445425890b23a (table test-table [id=e541e4a81d894f618da2009c64aa9960]) successfully deleted
I20260811 12:15:30.584805 11122 ts_tablet_manager.cc:1939] T b6f7835dcbc5493ba6c445425890b23a P d97463d1a263450fa63aa714db09e877: Deleting consensus metadata
I20260811 12:15:30.586273 10171 catalog_manager.cc:4841] TS d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545): tablet b6f7835dcbc5493ba6c445425890b23a (table test-table [id=e541e4a81d894f618da2009c64aa9960]) successfully deleted
I20260811 12:15:30.587249 11121 ts_tablet_manager.cc:1918] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:15:30.587833 11121 log.cc:1196] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/b6f7835dcbc5493ba6c445425890b23a
I20260811 12:15:30.588814 11121 ts_tablet_manager.cc:1939] T b6f7835dcbc5493ba6c445425890b23a P 7ca842b535a949f99111414c78e83bae: Deleting consensus metadata
I20260811 12:15:30.590149 10170 catalog_manager.cc:4841] TS 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297): tablet b6f7835dcbc5493ba6c445425890b23a (table test-table [id=e541e4a81d894f618da2009c64aa9960]) successfully deleted
I20260811 12:15:30.590818 10403 tablet_service.cc:1467] Processing CreateTablet for tablet ae8f28e521284002a22d287c30d41d71 (DEFAULT_TABLE table=test-table [id=b471228e682d486ab0d9b540f15d47c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:30.591361 10403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8f28e521284002a22d287c30d41d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:30.591349 10640 tablet_service.cc:1467] Processing CreateTablet for tablet ae8f28e521284002a22d287c30d41d71 (DEFAULT_TABLE table=test-table [id=b471228e682d486ab0d9b540f15d47c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:30.591889 10640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8f28e521284002a22d287c30d41d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:30.593715 10860 tablet_service.cc:1467] Processing CreateTablet for tablet ae8f28e521284002a22d287c30d41d71 (DEFAULT_TABLE table=test-table [id=b471228e682d486ab0d9b540f15d47c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:30.594363 10860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8f28e521284002a22d287c30d41d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:30.599650 11137 tablet_bootstrap.cc:492] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: Bootstrap starting.
I20260811 12:15:30.602173 11137 tablet_bootstrap.cc:654] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:30.602223 11136 tablet_bootstrap.cc:492] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: Bootstrap starting.
I20260811 12:15:30.604892 11136 tablet_bootstrap.cc:654] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:30.605110 11138 tablet_bootstrap.cc:492] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: Bootstrap starting.
I20260811 12:15:30.607694 11138 tablet_bootstrap.cc:654] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:30.613797 11137 tablet_bootstrap.cc:492] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: No bootstrap required, opened a new log
I20260811 12:15:30.613961 11138 tablet_bootstrap.cc:492] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: No bootstrap required, opened a new log
I20260811 12:15:30.614120 11137 ts_tablet_manager.cc:1397] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.003s
I20260811 12:15:30.614180 11138 ts_tablet_manager.cc:1397] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260811 12:15:30.615319 11137 raft_consensus.cc:348] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.615173 11138 raft_consensus.cc:348] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.615654 11138 raft_consensus.cc:374] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:30.615716 11137 raft_consensus.cc:374] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:30.615787 11138 raft_consensus.cc:729] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca842b535a949f99111414c78e83bae, State: Initialized, Role: FOLLOWER
I20260811 12:15:30.615799 11137 raft_consensus.cc:729] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee4528070b24536ae7bc91b362a1edb, State: Initialized, Role: FOLLOWER
I20260811 12:15:30.616115 11137 consensus_queue.cc:260] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.616242 11138 consensus_queue.cc:260] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.618600 11138 ts_tablet_manager.cc:1428] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 12:15:30.617606 11137 ts_tablet_manager.cc:1428] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:30.623003 11136 tablet_bootstrap.cc:492] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: No bootstrap required, opened a new log
I20260811 12:15:30.623350 11136 ts_tablet_manager.cc:1397] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.011s
I20260811 12:15:30.624266 11136 raft_consensus.cc:348] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.624585 11136 raft_consensus.cc:374] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:30.624675 11136 raft_consensus.cc:729] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97463d1a263450fa63aa714db09e877, State: Initialized, Role: FOLLOWER
I20260811 12:15:30.624946 11136 consensus_queue.cc:260] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.625555 11136 ts_tablet_manager.cc:1428] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:15:30.763768 11087 raft_consensus.cc:482] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:30.764036 11087 raft_consensus.cc:504] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.765118 11087 leader_election.cc:288] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545), bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419)
I20260811 12:15:30.766011 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8f28e521284002a22d287c30d41d71" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bee4528070b24536ae7bc91b362a1edb" is_pre_election: true
I20260811 12:15:30.765988 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8f28e521284002a22d287c30d41d71" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97463d1a263450fa63aa714db09e877" is_pre_election: true
I20260811 12:15:30.766363 10675 raft_consensus.cc:2454] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 0.
I20260811 12:15:30.766497 10435 raft_consensus.cc:2454] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 0.
I20260811 12:15:30.766909 10791 leader_election.cc:302] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca842b535a949f99111414c78e83bae, bee4528070b24536ae7bc91b362a1edb; no voters:
I20260811 12:15:30.767338 11087 raft_consensus.cc:2781] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:30.767570 11087 raft_consensus.cc:482] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:30.767731 11087 raft_consensus.cc:3037] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:30.770627 11087 raft_consensus.cc:504] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.771377 11087 leader_election.cc:288] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 election: Requested vote from peers d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545), bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419)
I20260811 12:15:30.772010 10435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8f28e521284002a22d287c30d41d71" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97463d1a263450fa63aa714db09e877"
I20260811 12:15:30.772078 10675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8f28e521284002a22d287c30d41d71" candidate_uuid: "7ca842b535a949f99111414c78e83bae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bee4528070b24536ae7bc91b362a1edb"
I20260811 12:15:30.772280 10435 raft_consensus.cc:3037] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:30.772452 10675 raft_consensus.cc:3037] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:30.776510 10675 raft_consensus.cc:2454] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 1.
I20260811 12:15:30.776824 10435 raft_consensus.cc:2454] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca842b535a949f99111414c78e83bae in term 1.
I20260811 12:15:30.777181 10791 leader_election.cc:302] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca842b535a949f99111414c78e83bae, bee4528070b24536ae7bc91b362a1edb; no voters:
I20260811 12:15:30.777628 11087 raft_consensus.cc:2781] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:30.777861 11087 raft_consensus.cc:686] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 1 LEADER]: Becoming Leader. State: Replica: 7ca842b535a949f99111414c78e83bae, State: Running, Role: LEADER
I20260811 12:15:30.778201 11087 consensus_queue.cc:237] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } }
I20260811 12:15:30.782047 10194 catalog_manager.cc:5495] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ca842b535a949f99111414c78e83bae (127.7.39.3). New cstate: current_term: 1 leader_uuid: "7ca842b535a949f99111414c78e83bae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ca842b535a949f99111414c78e83bae" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45297 } health_report { overall_health: HEALTHY } } }
I20260811 12:15:30.821194 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:30.813301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.7.39.3:45297)
0811 12:15:30.813494 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:30.813500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:30.814127 (+ 627us) server_negotiation.cc:407] Connection header received
0811 12:15:30.814355 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:30.814361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:30.814453 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:30.814555 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:30.815009 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.816433 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.818599 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:30.819525 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:30.819661 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:30.820289 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:30.820434 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:30.820526 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:30.820647 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 12:15:30.820749 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:15:30.833765 10434 raft_consensus.cc:1261] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Refusing update from remote peer 7ca842b535a949f99111414c78e83bae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:30.834738 11183 consensus_queue.cc:1035] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [LEADER]: Connected to new peer: Peer: permanent_uuid: "d97463d1a263450fa63aa714db09e877" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 12:15:30.845103 10675 raft_consensus.cc:1261] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Refusing update from remote peer 7ca842b535a949f99111414c78e83bae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:30.846010 11109 consensus_queue.cc:1035] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bee4528070b24536ae7bc91b362a1edb" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:15:30.864212 11084 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 12:15:32.460803 10194 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:32.461040 10194 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:32.465991 10194 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9eed0a07d56a4236a6aae0ed6908ddf7: Sending DeleteTablet for 3 replicas of tablet ae8f28e521284002a22d287c30d41d71
I20260811 12:15:32.466894 10640 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8f28e521284002a22d287c30d41d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:32 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:37174
I20260811 12:15:32.466998 10403 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8f28e521284002a22d287c30d41d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:32 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:44158
I20260811 12:15:32.467391 10860 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8f28e521284002a22d287c30d41d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:32 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:43622
I20260811 12:15:32.468732 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10296
I20260811 12:15:32.469575 11220 tablet_replica.cc:330] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: stopping tablet replica
I20260811 12:15:32.469862 11220 raft_consensus.cc:2229] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:32.470099 11219 tablet_replica.cc:330] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: stopping tablet replica
I20260811 12:15:32.470294 11220 raft_consensus.cc:2258] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:32.470367 11219 raft_consensus.cc:2229] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:32.470788 11219 raft_consensus.cc:2258] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:32.471977 11219 ts_tablet_manager.cc:1905] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:32.480834 11219 ts_tablet_manager.cc:1918] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:15:32.480976 11219 log.cc:1196] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/ae8f28e521284002a22d287c30d41d71
I20260811 12:15:32.481618 11219 ts_tablet_manager.cc:1939] T ae8f28e521284002a22d287c30d41d71 P bee4528070b24536ae7bc91b362a1edb: Deleting consensus metadata
I20260811 12:15:32.481875 11220 ts_tablet_manager.cc:1905] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:32.482897 10173 catalog_manager.cc:4841] TS bee4528070b24536ae7bc91b362a1edb (127.7.39.2:35419): tablet ae8f28e521284002a22d287c30d41d71 (table test-table [id=b471228e682d486ab0d9b540f15d47c5]) successfully deleted
I20260811 12:15:32.484560 11221 tablet_replica.cc:330] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: stopping tablet replica
I20260811 12:15:32.484752 11221 raft_consensus.cc:2229] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:15:32.485085 11221 raft_consensus.cc:2258] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:32.491515 11220 ts_tablet_manager.cc:1918] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:15:32.491705 11220 log.cc:1196] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/ae8f28e521284002a22d287c30d41d71
I20260811 12:15:32.492113 11220 ts_tablet_manager.cc:1939] T ae8f28e521284002a22d287c30d41d71 P d97463d1a263450fa63aa714db09e877: Deleting consensus metadata
I20260811 12:15:32.493403 10171 catalog_manager.cc:4841] TS d97463d1a263450fa63aa714db09e877 (127.7.39.1:44545): tablet ae8f28e521284002a22d287c30d41d71 (table test-table [id=b471228e682d486ab0d9b540f15d47c5]) successfully deleted
I20260811 12:15:32.493775 11221 ts_tablet_manager.cc:1905] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:15:32 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450530, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:32.501060 11221 ts_tablet_manager.cc:1918] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:15:32.501382 11221 log.cc:1196] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/ae8f28e521284002a22d287c30d41d71
I20260811 12:15:32.501997 11221 ts_tablet_manager.cc:1939] T ae8f28e521284002a22d287c30d41d71 P 7ca842b535a949f99111414c78e83bae: Deleting consensus metadata
I20260811 12:15:32.503206 10170 catalog_manager.cc:4841] TS 7ca842b535a949f99111414c78e83bae (127.7.39.3:45297): tablet ae8f28e521284002a22d287c30d41d71 (table test-table [id=b471228e682d486ab0d9b540f15d47c5]) successfully deleted
I20260811 12:15:32.509302 11227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:32.470911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.7.39.1:44545)
0811 12:15:32.477246 (+ 6335us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:32.477252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:32.480877 (+ 3625us) server_negotiation.cc:407] Connection header received
0811 12:15:32.481452 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:32.481460 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:32.481574 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:32.481659 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:32.484084 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:32.485569 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:32.488040 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:32.488776 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:32.488938 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:32.500990 (+ 12052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:32.501020 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:32.501044 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:32.501106 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:32.505683 (+ 4577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:32.506643 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:32.506652 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:32.506655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:32.506933 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:32.507309 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:32.507313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:32.507315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:32.507855 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:32.508225 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:32.508598 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 12:15:32.508789 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6180,"thread_start_us":6110,"threads_started":1}
W20260811 12:15:32.510227 10507 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36844
W20260811 12:15:32.511145 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10296: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:32.511399 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10296
I20260811 12:15:32.541239 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10521
Aug 11 12:15:32 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450530, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:15:32.559147 11228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:32.542120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.7.39.2:35419)
0811 12:15:32.542788 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:32.542793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:32.542810 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:15:32.542868 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:32.542872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:32.542934 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:32.543026 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:32.544274 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:32.545061 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:32.546406 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:32.546831 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:32.546934 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:32.551828 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:32.551852 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:32.551877 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:32.551944 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:32.556151 (+ 4207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:32.556992 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:32.557002 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:32.557005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:32.557244 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:32.557644 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:32.557649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:32.557651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:32.558075 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:32.558348 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:32.558571 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 12:15:32.558787 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":381,"threads_started":1}
W20260811 12:15:32.559892 10745 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53652
W20260811 12:15:32.560554 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10521: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:32.560745 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10521
I20260811 12:15:32.646044 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10754
Aug 11 12:15:32 dist-test-slave-rmhg krb5kdc[10129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450530, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:15:32.661923 11229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:32.646834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.7.39.3:45297)
0811 12:15:32.647384 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:32.647389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:32.647405 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:15:32.647473 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:32.647479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:32.647643 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:32.647740 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:32.649112 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:32.649928 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:32.651139 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:32.651545 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:32.651635 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:32.655890 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:32.655907 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:32.655926 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:32.655963 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:32.659001 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:32.659744 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:32.659752 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:32.659755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:32.659893 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:32.660270 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:32.660274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:32.660276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:32.660655 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:32.660954 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:32.661338 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 12:15:32.661582 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":255,"threads_started":1}
W20260811 12:15:32.662622 10963 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51110
W20260811 12:15:32.663254 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10754: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:32.663434 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10754
I20260811 12:15:32.704787 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10148
I20260811 12:15:32.706147 10254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:32.826216 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10148
2026-08-11T12:15:32Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8981 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:32 dist-test-slave-rmhg krb5kdc[11234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:32 dist-test-slave-rmhg krb5kdc[11234](info): set up 2 sockets
Aug 11 12:15:32 dist-test-slave-rmhg krb5kdc[11234](info): commencing operation
krb5kdc: starting...
W20260811 12:15:34.056715 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:34 dist-test-slave-rmhg krb5kdc[11234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:34.291117 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:42495
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:42495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:34.659410 11250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:34.659878 11250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:34.660028 11250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:34.667702 11250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:34.667845 11250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:34.667893 11250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:34.667951 11250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:34.667995 11250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:34.678344 11250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:42495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:42495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:34.679737 11250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:34.681389 11250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:34.694114 11256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:34.694132 11260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:34.694437 11257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:34.695140 11250 server_base.cc:1034] running on GCE node
I20260811 12:15:34.696103 11250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:34.697822 11250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:34.699061 11250 hybrid_clock.cc:648] HybridClock initialized: now 1786450534699006 us; error 45 us; skew 500 ppm
I20260811 12:15:34.699455 11250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:34 dist-test-slave-rmhg krb5kdc[11234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450534, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:34.703017 11250 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:34.704790 11250 webserver.cc:460] Webserver started at http://127.7.39.62:33073/ using document root <none> and password file <none>
I20260811 12:15:34.705729 11250 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:34.705853 11250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:34.706195 11250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:34.709564 11250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "a81378bfe83441cda0a4a27f01d5d945"
format_stamp: "Formatted at 2026-08-11 12:15:34 on dist-test-slave-rmhg"
server_key: "3fbe8b7b24beca7a89e1e2e3d8610ead"
server_key_iv: "b08301cf6bd175e559bcb1e9eb495291"
server_key_version: "encryptionkey@0"
I20260811 12:15:34.710444 11250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "a81378bfe83441cda0a4a27f01d5d945"
format_stamp: "Formatted at 2026-08-11 12:15:34 on dist-test-slave-rmhg"
server_key: "3fbe8b7b24beca7a89e1e2e3d8610ead"
server_key_iv: "b08301cf6bd175e559bcb1e9eb495291"
server_key_version: "encryptionkey@0"
I20260811 12:15:34.716439 11250 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 12:15:34.720669 11269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:34.722361 11250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:34.722550 11250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "a81378bfe83441cda0a4a27f01d5d945"
format_stamp: "Formatted at 2026-08-11 12:15:34 on dist-test-slave-rmhg"
server_key: "3fbe8b7b24beca7a89e1e2e3d8610ead"
server_key_iv: "b08301cf6bd175e559bcb1e9eb495291"
server_key_version: "encryptionkey@0"
I20260811 12:15:34.722795 11250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:34.740782 11250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:34.745836 11250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:34.746142 11250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:34.765870 11250 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:42495
I20260811 12:15:34.765894 11360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:42495 every 8 connection(s)
I20260811 12:15:34.767427 11250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:34.771561 11361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:34.775743 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11250
I20260811 12:15:34.775965 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:34.776582 7324 external_mini_cluster.cc:1428] Setting key 1594a1510e94e050a3cbc8c9f24b2487
I20260811 12:15:34.783344 11361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945: Bootstrap starting.
I20260811 12:15:34.787597 11361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:34.788772 11361 log.cc:824] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:34.791990 11361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945: No bootstrap required, opened a new log
Aug 11 12:15:34 dist-test-slave-rmhg krb5kdc[11234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:34.801705 11361 raft_consensus.cc:348] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81378bfe83441cda0a4a27f01d5d945" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42495 } }
I20260811 12:15:34.802110 11361 raft_consensus.cc:374] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:34.802211 11361 raft_consensus.cc:729] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a81378bfe83441cda0a4a27f01d5d945, State: Initialized, Role: FOLLOWER
I20260811 12:15:34.802305 11364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:34.778684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.7.39.62:42495)
0811 12:15:34.779552 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:34.779565 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:34.779625 (+ 60us) server_negotiation.cc:407] Connection header received
0811 12:15:34.781815 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:34.781846 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:34.782346 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:34.782771 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:34.784514 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:34.786069 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:34.787502 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:34.788679 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:34.788877 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:34.793647 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:34.793656 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:34.793674 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:34.793716 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:34.797288 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:34.798280 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:34.798287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:34.798296 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:34.798506 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:34.798965 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:34.798970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:34.798972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:34.799720 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:34.799996 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:34.800549 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 12:15:34.801070 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":398,"threads_started":1}
I20260811 12:15:34.802964 11361 consensus_queue.cc:260] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81378bfe83441cda0a4a27f01d5d945" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42495 } }
I20260811 12:15:34.803283 11361 raft_consensus.cc:388] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:34.803424 11361 raft_consensus.cc:482] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:34.803609 11361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:34.881229 11361 raft_consensus.cc:504] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81378bfe83441cda0a4a27f01d5d945" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42495 } }
I20260811 12:15:34.882048 11361 leader_election.cc:302] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a81378bfe83441cda0a4a27f01d5d945; no voters:
I20260811 12:15:34.882829 11361 leader_election.cc:288] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:34.882969 11368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:34.886339 11368 raft_consensus.cc:686] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [term 1 LEADER]: Becoming Leader. State: Replica: a81378bfe83441cda0a4a27f01d5d945, State: Running, Role: LEADER
I20260811 12:15:34.887001 11368 consensus_queue.cc:237] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81378bfe83441cda0a4a27f01d5d945" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42495 } }
I20260811 12:15:34.887238 11361 sys_catalog.cc:564] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:34.890817 11371 sys_catalog.cc:455] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a81378bfe83441cda0a4a27f01d5d945. Latest consensus state: current_term: 1 leader_uuid: "a81378bfe83441cda0a4a27f01d5d945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81378bfe83441cda0a4a27f01d5d945" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42495 } } }
I20260811 12:15:34.891317 11371 sys_catalog.cc:458] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:34.891276 11370 sys_catalog.cc:455] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a81378bfe83441cda0a4a27f01d5d945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81378bfe83441cda0a4a27f01d5d945" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42495 } } }
I20260811 12:15:34.891597 11370 sys_catalog.cc:458] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:34.892346 11388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:34.898547 11388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:34.910843 11388 catalog_manager.cc:1347] Generated new cluster ID: e7d87a24c083422ab7b4b86d9303e450
I20260811 12:15:34.910990 11388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:34.926692 11388 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:34.928279 11388 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:34.939253 11388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a81378bfe83441cda0a4a27f01d5d945: Generated new TSK 0
I20260811 12:15:34.940189 11388 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
W20260811 12:15:35.761447 7324 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.39.1: real 0.814s user 0.000s sys 0.016s
I20260811 12:15:35.762578 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42495
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:36.107782 11408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:36.108110 11408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:36.108248 11408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:36.115711 11408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:36.115868 11408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:36.116031 11408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:36.126494 11408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:36.128131 11408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:36.129590 11408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:36.143502 11415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:36.143548 11418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:36.143474 11414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:36.144367 11408 server_base.cc:1034] running on GCE node
I20260811 12:15:36.145009 11408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:36.146047 11408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:36.147258 11408 hybrid_clock.cc:648] HybridClock initialized: now 1786450536147201 us; error 48 us; skew 500 ppm
I20260811 12:15:36.147614 11408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:36 dist-test-slave-rmhg krb5kdc[11234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450536, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:36.150894 11408 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:15:36.152412 11408 webserver.cc:460] Webserver started at http://127.7.39.1:33967/ using document root <none> and password file <none>
I20260811 12:15:36.153117 11408 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:36.153276 11408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:36.153602 11408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:36.156571 11408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "4b0c57cf6f984120aaa1a0b4b1544644"
format_stamp: "Formatted at 2026-08-11 12:15:36 on dist-test-slave-rmhg"
server_key: "884451db478e2cac1c9087ba01596898"
server_key_iv: "04869a8b05a1e0c68e96354d08746e35"
server_key_version: "encryptionkey@0"
I20260811 12:15:36.157416 11408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "4b0c57cf6f984120aaa1a0b4b1544644"
format_stamp: "Formatted at 2026-08-11 12:15:36 on dist-test-slave-rmhg"
server_key: "884451db478e2cac1c9087ba01596898"
server_key_iv: "04869a8b05a1e0c68e96354d08746e35"
server_key_version: "encryptionkey@0"
I20260811 12:15:36.162810 11408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:15:36.166632 11430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:36.168128 11408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 12:15:36.168275 11408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "4b0c57cf6f984120aaa1a0b4b1544644"
format_stamp: "Formatted at 2026-08-11 12:15:36 on dist-test-slave-rmhg"
server_key: "884451db478e2cac1c9087ba01596898"
server_key_iv: "04869a8b05a1e0c68e96354d08746e35"
server_key_version: "encryptionkey@0"
I20260811 12:15:36.168455 11408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:36.194674 11408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:36.199393 11408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:36.199671 11408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:36.200667 11408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:36.202117 11408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:36.202195 11408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:36.202289 11408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:36.202332 11408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:36.236188 11408 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:42615
I20260811 12:15:36.236241 11593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:42615 every 8 connection(s)
I20260811 12:15:36.237600 11408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:36.239652 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11408
I20260811 12:15:36.239912 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:36.240795 7324 external_mini_cluster.cc:1428] Setting key a26e7bf16da4068636baad902b7342b2
Aug 11 12:15:36 dist-test-slave-rmhg krb5kdc[11234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450536, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:36.261615 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:36.240602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:39665 (local address 127.7.39.62:42495)
0811 12:15:36.241105 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:36.241110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:36.242613 (+ 1503us) server_negotiation.cc:407] Connection header received
0811 12:15:36.244768 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:36.244773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:36.244861 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:36.244958 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:36.246779 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:36.247529 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:36.249509 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:36.249957 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:36.250080 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:36.254694 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:36.254703 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:36.254707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:36.254762 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:36.258016 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:36.258939 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:36.258945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:36.258949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:36.259125 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:36.259572 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:36.259577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:36.259581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:36.259951 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:36.260125 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:36.260933 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 12:15:36.261231 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":271,"threads_started":1}
I20260811 12:15:36.262588 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:36.241446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42495 (local address 127.7.39.1:39665)
0811 12:15:36.242406 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:36.242477 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:36.244465 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:36.245319 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:36.245332 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:36.245950 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:36.246579 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:36.246600 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:36.247708 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:36.247712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:36.249338 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:36.249355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:36.250215 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:36.250223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:36.250476 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:36.251256 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:36.251289 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:36.254321 (+ 3032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:36.258209 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:36.258218 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:36.258235 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:36.258757 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:36.259230 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:36.259234 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:36.259236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:36.259448 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:36.260094 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:36.260100 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:36.260474 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 12:15:36.260767 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 12:15:36.261320 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":354,"threads_started":1}
I20260811 12:15:36.264324 11594 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42495
I20260811 12:15:36.264680 11594 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
I20260811 12:15:36.265918 11594 heartbeater.cc:511] Master 127.7.39.62:42495 requested a full tablet report, sending...
I20260811 12:15:36.268204 11309 ts_manager.cc:194] Registered new tserver with Master: 4b0c57cf6f984120aaa1a0b4b1544644 (127.7.39.1:42615)
I20260811 12:15:36.269825 11309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:39665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
W20260811 12:15:36.549317 7324 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.39.2: real 0.305s user 0.005s sys 0.011s
I20260811 12:15:36.550124 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42495
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:36.893673 11606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:36.893980 11606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:36.894083 11606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:36.901690 11606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:36.901818 11606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:36.901937 11606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:36.912091 11606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:36.913486 11606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:36.914811 11606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:36.928644 11616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:36.928558 11619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:36.928596 11617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:36.929471 11606 server_base.cc:1034] running on GCE node
I20260811 12:15:36.930142 11606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:36.931195 11606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:36.932400 11606 hybrid_clock.cc:648] HybridClock initialized: now 1786450536932342 us; error 49 us; skew 500 ppm
I20260811 12:15:36.932730 11606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:36 dist-test-slave-rmhg krb5kdc[11234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450536, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:36.935846 11606 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:15:36.937381 11606 webserver.cc:460] Webserver started at http://127.7.39.2:41345/ using document root <none> and password file <none>
I20260811 12:15:36.938084 11606 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:36.938221 11606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:36.938555 11606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:36.941808 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "25da36aed1f14837917e38e7b65fbf87"
format_stamp: "Formatted at 2026-08-11 12:15:36 on dist-test-slave-rmhg"
server_key: "5d4ec1e705718084771ea9c031e46d4d"
server_key_iv: "d14b8167f075b967acf1e162a9c36db0"
server_key_version: "encryptionkey@0"
I20260811 12:15:36.942679 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "25da36aed1f14837917e38e7b65fbf87"
format_stamp: "Formatted at 2026-08-11 12:15:36 on dist-test-slave-rmhg"
server_key: "5d4ec1e705718084771ea9c031e46d4d"
server_key_iv: "d14b8167f075b967acf1e162a9c36db0"
server_key_version: "encryptionkey@0"
I20260811 12:15:36.948202 11606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 12:15:36.951828 11631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:36.953284 11606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:36.953472 11606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "25da36aed1f14837917e38e7b65fbf87"
format_stamp: "Formatted at 2026-08-11 12:15:36 on dist-test-slave-rmhg"
server_key: "5d4ec1e705718084771ea9c031e46d4d"
server_key_iv: "d14b8167f075b967acf1e162a9c36db0"
server_key_version: "encryptionkey@0"
I20260811 12:15:36.953673 11606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:36.994565 11606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:36.999109 11606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:36.999433 11606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:37.000452 11606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:37.002063 11606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:37.002147 11606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:37.002305 11606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:37.002381 11606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:37.034449 11606 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:41593
I20260811 12:15:37.034495 11806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:41593 every 8 connection(s)
I20260811 12:15:37.035725 11606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:37.044449 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11606
I20260811 12:15:37.044659 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:37.045082 7324 external_mini_cluster.cc:1428] Setting key 7764ebcd2f5baaae5d3483ea1bce4767
Aug 11 12:15:37 dist-test-slave-rmhg krb5kdc[11234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450536, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:37.056680 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:37.038342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:34229 (local address 127.7.39.62:42495)
0811 12:15:37.038810 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:37.038815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:37.039702 (+ 887us) server_negotiation.cc:407] Connection header received
0811 12:15:37.041601 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:37.041607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:37.041717 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:37.041817 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:37.043425 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:37.044200 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:37.045982 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:37.046635 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:37.046780 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:37.050852 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:37.050857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:37.050860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:37.050893 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:37.053462 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:37.054277 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:37.054280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:37.054283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:37.054407 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:37.054892 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:37.054897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:37.054901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:37.055269 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:37.055408 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:37.056088 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 12:15:37.056331 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":358,"thread_start_us":243,"threads_started":1}
I20260811 12:15:37.057583 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:37.038824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42495 (local address 127.7.39.2:34229)
0811 12:15:37.039520 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:37.039560 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:37.041319 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:37.042082 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:37.042091 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:37.042643 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:37.043238 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:37.043257 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:37.044365 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:37.044369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:37.045823 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:37.045838 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:37.046885 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:37.046893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:37.047173 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:37.047885 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:37.047917 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:37.050590 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:37.053594 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:37.053603 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:37.053620 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:37.054172 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:37.054504 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:37.054508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:37.054511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:37.054756 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:37.055385 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:37.055391 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:37.055717 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 12:15:37.055939 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 12:15:37.056350 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":281,"threads_started":1}
I20260811 12:15:37.059077 11809 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42495
I20260811 12:15:37.059433 11809 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:37.060384 11809 heartbeater.cc:511] Master 127.7.39.62:42495 requested a full tablet report, sending...
I20260811 12:15:37.062021 11309 ts_manager.cc:194] Registered new tserver with Master: 25da36aed1f14837917e38e7b65fbf87 (127.7.39.2:41593)
I20260811 12:15:37.062870 11309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:34229
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:15:37.096884 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42495
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:15:37.273211 11594 heartbeater.cc:503] Master 127.7.39.62:42495 was elected leader, sending a full tablet report...
W20260811 12:15:37.440624 11818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:37.440950 11818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:37.441048 11818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:37.448592 11818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:37.448729 11818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:37.448851 11818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:37.459748 11818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:37.461196 11818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:37.462682 11818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:37.476730 11828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:37.476668 11831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:37.476776 11829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:37.482825 11818 server_base.cc:1034] running on GCE node
I20260811 12:15:37.483526 11818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:37.484611 11818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:37.485814 11818 hybrid_clock.cc:648] HybridClock initialized: now 1786450537485751 us; error 55 us; skew 500 ppm
I20260811 12:15:37.486142 11818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:37 dist-test-slave-rmhg krb5kdc[11234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450537, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:37.489616 11818 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:15:37.491276 11818 webserver.cc:460] Webserver started at http://127.7.39.3:33109/ using document root <none> and password file <none>
I20260811 12:15:37.492049 11818 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:37.492203 11818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:37.492543 11818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:37.495756 11818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "a928c085c6e84ebea1d6b21a77bbd673"
format_stamp: "Formatted at 2026-08-11 12:15:37 on dist-test-slave-rmhg"
server_key: "bc2a898696a2cd03d97ab2ea008ea7e9"
server_key_iv: "8fc8c2db53416109e56462d43760eaac"
server_key_version: "encryptionkey@0"
I20260811 12:15:37.496594 11818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "a928c085c6e84ebea1d6b21a77bbd673"
format_stamp: "Formatted at 2026-08-11 12:15:37 on dist-test-slave-rmhg"
server_key: "bc2a898696a2cd03d97ab2ea008ea7e9"
server_key_iv: "8fc8c2db53416109e56462d43760eaac"
server_key_version: "encryptionkey@0"
I20260811 12:15:37.502554 11818 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 12:15:37.506721 11844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:37.508313 11818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:37.508515 11818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "a928c085c6e84ebea1d6b21a77bbd673"
format_stamp: "Formatted at 2026-08-11 12:15:37 on dist-test-slave-rmhg"
server_key: "bc2a898696a2cd03d97ab2ea008ea7e9"
server_key_iv: "8fc8c2db53416109e56462d43760eaac"
server_key_version: "encryptionkey@0"
I20260811 12:15:37.508726 11818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:37.537530 11818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:37.542429 11818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:37.542722 11818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:37.543769 11818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:37.545307 11818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:37.545390 11818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:37.545488 11818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:37.545532 11818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:37.579655 11818 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:39133
I20260811 12:15:37.579771 12029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:39133 every 8 connection(s)
I20260811 12:15:37.581079 11818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:37.590277 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11818
I20260811 12:15:37.590453 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:37.590895 7324 external_mini_cluster.cc:1428] Setting key 9600a3acbc88e729f35098c02aa48dc3
Aug 11 12:15:37 dist-test-slave-rmhg krb5kdc[11234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450537, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:37.602795 12032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:37.583872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:38291 (local address 127.7.39.62:42495)
0811 12:15:37.584492 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:37.584499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:37.585604 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 12:15:37.587579 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:37.587584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:37.587664 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:37.587777 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:37.589628 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:37.590447 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:37.592439 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:37.593057 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:37.593231 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:37.597060 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:37.597065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:37.597069 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:37.597106 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:37.599672 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:37.600549 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:37.600554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:37.600557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:37.600695 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:37.601079 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:37.601081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:37.601084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:37.601408 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:37.601562 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:37.602268 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 12:15:37.602465 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":242,"threads_started":1}
I20260811 12:15:37.603837 12033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:37.584533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42495 (local address 127.7.39.3:38291)
0811 12:15:37.585392 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:37.585453 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:37.587254 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:37.588095 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:37.588105 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:37.588711 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:37.589424 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:37.589449 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:37.590631 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:37.590635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:37.592269 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:37.592287 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:37.593339 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:37.593348 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:37.593688 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:37.594500 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:37.594525 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:37.596804 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:37.599846 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:37.599855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:37.599872 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:37.600370 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:37.600783 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:37.600786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:37.600789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:37.600990 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:37.601518 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:37.601525 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:37.601907 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 12:15:37.602150 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 12:15:37.602592 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":307,"threads_started":1}
I20260811 12:15:37.605450 12030 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42495
I20260811 12:15:37.605794 12030 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:37.606621 12030 heartbeater.cc:511] Master 127.7.39.62:42495 requested a full tablet report, sending...
I20260811 12:15:37.608139 11309 ts_manager.cc:194] Registered new tserver with Master: a928c085c6e84ebea1d6b21a77bbd673 (127.7.39.3:39133)
I20260811 12:15:37.609002 11309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:38291
I20260811 12:15:37.619212 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:15:37.658097 12032 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:15:37.655451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.7.39.62:42495)
0811 12:15:37.655650 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:37.655655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:37.656119 (+ 464us) server_negotiation.cc:407] Connection header received
0811 12:15:37.656281 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:37.656285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:37.656359 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:37.656461 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:37.657869 (+ 1408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37210: BlockingRecv error: recv got EOF from 127.0.0.1:37210 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:15:37.661804 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11408
W20260811 12:15:37.664686 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11408: Not authorized: Client connection negotiation failed: client connection to 127.7.39.1:42615: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:15:37.664899 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11408
I20260811 12:15:37.689697 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11606
W20260811 12:15:37.692580 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11606: Not authorized: Client connection negotiation failed: client connection to 127.7.39.2:41593: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:15:37.692762 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11606
I20260811 12:15:37.722059 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11818
W20260811 12:15:37.725111 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11818: Not authorized: Client connection negotiation failed: client connection to 127.7.39.3:39133: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:15:37.725308 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11818
I20260811 12:15:37.749272 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11250
I20260811 12:15:37.750581 11352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:37.855536 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11250
2026-08-11T12:15:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (5030 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:37 dist-test-slave-rmhg krb5kdc[12057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:37 dist-test-slave-rmhg krb5kdc[12057](info): set up 2 sockets
Aug 11 12:15:37 dist-test-slave-rmhg krb5kdc[12057](info): commencing operation
krb5kdc: starting...
W20260811 12:15:39.086109 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:39 dist-test-slave-rmhg krb5kdc[12057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:39.324359 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35737
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:33495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:35737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:39.664346 12075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:39.664705 12075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:39.664825 12075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:39.672549 12075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:39.672675 12075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:39.672748 12075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:39.672834 12075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:39.672899 12075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:39.683053 12075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:33495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:35737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:39.684407 12075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:39.685854 12075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:39.698076 12085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:39.698256 12082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:39.698258 12087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:39.699148 12075 server_base.cc:1034] running on GCE node
I20260811 12:15:39.700029 12075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:39.701568 12075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:39.702787 12075 hybrid_clock.cc:648] HybridClock initialized: now 1786450539702735 us; error 47 us; skew 500 ppm
I20260811 12:15:39.703179 12075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:39 dist-test-slave-rmhg krb5kdc[12057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450539, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:39.706492 12075 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:39.708056 12075 webserver.cc:460] Webserver started at http://127.7.39.62:34637/ using document root <none> and password file <none>
I20260811 12:15:39.708760 12075 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:39.708887 12075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:39.709259 12075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:39.712359 12075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "0e1f59fc14e5482b9a85c8937b71a51e"
format_stamp: "Formatted at 2026-08-11 12:15:39 on dist-test-slave-rmhg"
server_key: "dc31c9e569c3e4dbdcb42faa4e6a1cf3"
server_key_iv: "85cba754f8f87a8137e12875b7f09132"
server_key_version: "encryptionkey@0"
I20260811 12:15:39.713145 12075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "0e1f59fc14e5482b9a85c8937b71a51e"
format_stamp: "Formatted at 2026-08-11 12:15:39 on dist-test-slave-rmhg"
server_key: "dc31c9e569c3e4dbdcb42faa4e6a1cf3"
server_key_iv: "85cba754f8f87a8137e12875b7f09132"
server_key_version: "encryptionkey@0"
I20260811 12:15:39.718604 12075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:15:39.722231 12095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:39.723676 12075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:15:39.723855 12075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "0e1f59fc14e5482b9a85c8937b71a51e"
format_stamp: "Formatted at 2026-08-11 12:15:39 on dist-test-slave-rmhg"
server_key: "dc31c9e569c3e4dbdcb42faa4e6a1cf3"
server_key_iv: "85cba754f8f87a8137e12875b7f09132"
server_key_version: "encryptionkey@0"
I20260811 12:15:39.724056 12075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:39.735764 12075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:39.739897 12075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:39.740193 12075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:39.758006 12075 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:35737
I20260811 12:15:39.758037 12172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:35737 every 8 connection(s)
I20260811 12:15:39.759505 12075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:39.763360 12173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:39.767027 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 12075
I20260811 12:15:39.767205 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:39.767896 7324 external_mini_cluster.cc:1428] Setting key f61be3cf43e9cef1f69e0580644036d9
I20260811 12:15:39.776968 12173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e: Bootstrap starting.
I20260811 12:15:39.781203 12173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:39.782717 12173 log.cc:824] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:15:39 dist-test-slave-rmhg krb5kdc[12057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:39.786391 12173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e: No bootstrap required, opened a new log
I20260811 12:15:39.792893 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:39.769997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43954 (local address 127.7.39.62:35737)
0811 12:15:39.770852 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:39.770866 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:39.770913 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:15:39.772625 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:39.772652 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:39.773185 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:39.773593 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:39.775550 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:39.776941 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:39.778826 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:39.779618 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:39.779786 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:39.784856 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:39.784866 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:39.784884 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:39.784925 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:39.788242 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:39.789120 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:39.789128 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:39.789140 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:39.789391 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:39.789799 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:39.789807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:39.789809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:39.790433 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:39.790764 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:39.791278 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 12:15:39.791744 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":382,"threads_started":1}
I20260811 12:15:39.794669 12173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1f59fc14e5482b9a85c8937b71a51e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35737 } }
I20260811 12:15:39.795066 12173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:39.795171 12173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1f59fc14e5482b9a85c8937b71a51e, State: Initialized, Role: FOLLOWER
I20260811 12:15:39.795883 12173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1f59fc14e5482b9a85c8937b71a51e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35737 } }
I20260811 12:15:39.796169 12173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:39.796330 12173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:39.796531 12173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:39.797255 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:15:39.799809 12173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1f59fc14e5482b9a85c8937b71a51e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35737 } }
I20260811 12:15:39.800462 12173 leader_election.cc:302] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e1f59fc14e5482b9a85c8937b71a51e; no voters:
I20260811 12:15:39.801204 12173 leader_election.cc:288] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:39.801321 12178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:39.803576 12173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:39.808530 12178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [term 1 LEADER]: Becoming Leader. State: Replica: 0e1f59fc14e5482b9a85c8937b71a51e, State: Running, Role: LEADER
I20260811 12:15:39.809396 12178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1f59fc14e5482b9a85c8937b71a51e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35737 } }
I20260811 12:15:39.814352 12181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e1f59fc14e5482b9a85c8937b71a51e. Latest consensus state: current_term: 1 leader_uuid: "0e1f59fc14e5482b9a85c8937b71a51e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1f59fc14e5482b9a85c8937b71a51e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35737 } } }
I20260811 12:15:39.814821 12181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:39.825760 12180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e1f59fc14e5482b9a85c8937b71a51e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1f59fc14e5482b9a85c8937b71a51e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35737 } } }
I20260811 12:15:39.826119 12180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e [sys.catalog]: This master's current role is: LEADER
W20260811 12:15:39.830125 12202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:15:39.830305 12202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:15:39.830518 12204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:39.832522 12204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:39.841979 12204 catalog_manager.cc:1347] Generated new cluster ID: 71ece39206a34208beafeea5b6589ef2
I20260811 12:15:39.842123 12204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:39.864825 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:39.854498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.7.39.62:35737)
0811 12:15:39.854712 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:39.854720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:39.855267 (+ 547us) server_negotiation.cc:407] Connection header received
0811 12:15:39.855444 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:39.855450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:39.855549 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:39.855692 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:39.856906 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:39.858128 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:39.860057 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:39.860739 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:39.860864 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:39.862595 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:39.862603 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:39.862609 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:39.863797 (+ 1188us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:39.863899 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:39.863933 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:39.864069 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:39.864301 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 12:15:39.864451 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:15:39.866721 12204 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:39.868961 12204 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:39.878720 12204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e1f59fc14e5482b9a85c8937b71a51e: Generated new TSK 0
I20260811 12:15:39.879863 12204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:15:39.892254 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12075
I20260811 12:15:39.893852 12165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:39.987433 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 12075
2026-08-11T12:15:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2126 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:40 dist-test-slave-rmhg krb5kdc[12221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:40 dist-test-slave-rmhg krb5kdc[12221](info): set up 2 sockets
Aug 11 12:15:40 dist-test-slave-rmhg krb5kdc[12221](info): commencing operation
krb5kdc: starting...
W20260811 12:15:41.207928 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:41 dist-test-slave-rmhg krb5kdc[12221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:41.440183 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:41053
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:41053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:41.774463 12241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:41.774808 12241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:41.774900 12241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:41.782799 12241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:41.782931 12241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:41.782977 12241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:41.783036 12241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:41.783082 12241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:41.793272 12241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:41053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:41053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:41.794672 12241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:41.796018 12241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:41.808678 12251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:41.808595 12252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:41.808602 12254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:41.809743 12241 server_base.cc:1034] running on GCE node
I20260811 12:15:41.810659 12241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:41.812260 12241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:41.813472 12241 hybrid_clock.cc:648] HybridClock initialized: now 1786450541813413 us; error 53 us; skew 500 ppm
I20260811 12:15:41.813845 12241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:41 dist-test-slave-rmhg krb5kdc[12221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450541, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:41.817093 12241 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:41.818794 12241 webserver.cc:460] Webserver started at http://127.7.39.62:45709/ using document root <none> and password file <none>
I20260811 12:15:41.819491 12241 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:41.819609 12241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:41.819896 12241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:41.823122 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "6c02285b11d0469c9274fd28141ab37c"
format_stamp: "Formatted at 2026-08-11 12:15:41 on dist-test-slave-rmhg"
server_key: "ca6ac5d77a0debe4111a067db34d432d"
server_key_iv: "d9978837bc9b32c21065eb2a3e075aa4"
server_key_version: "encryptionkey@0"
I20260811 12:15:41.823896 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "6c02285b11d0469c9274fd28141ab37c"
format_stamp: "Formatted at 2026-08-11 12:15:41 on dist-test-slave-rmhg"
server_key: "ca6ac5d77a0debe4111a067db34d432d"
server_key_iv: "d9978837bc9b32c21065eb2a3e075aa4"
server_key_version: "encryptionkey@0"
I20260811 12:15:41.829487 12241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:15:41.833226 12266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:41.834769 12241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:41.834955 12241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "6c02285b11d0469c9274fd28141ab37c"
format_stamp: "Formatted at 2026-08-11 12:15:41 on dist-test-slave-rmhg"
server_key: "ca6ac5d77a0debe4111a067db34d432d"
server_key_iv: "d9978837bc9b32c21065eb2a3e075aa4"
server_key_version: "encryptionkey@0"
I20260811 12:15:41.835168 12241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:41.860056 12241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:41.864544 12241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:41.864861 12241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:41.883306 12241 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:41053
I20260811 12:15:41.883335 12329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:41053 every 8 connection(s)
I20260811 12:15:41.884765 12241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:41.888741 12330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:41.892509 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 12241
I20260811 12:15:41.892765 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:41.893669 7324 external_mini_cluster.cc:1428] Setting key e040effd5027c1ce3b302c5799676907
I20260811 12:15:41.902778 12330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c: Bootstrap starting.
I20260811 12:15:41.907011 12330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:41.908492 12330 log.cc:824] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:15:41 dist-test-slave-rmhg krb5kdc[12221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:41.911756 12330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c: No bootstrap required, opened a new log
I20260811 12:15:41.917840 12334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:41.895997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.7.39.62:41053)
0811 12:15:41.896819 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:41.896833 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:41.896880 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:15:41.898553 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:41.898580 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:41.899034 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:41.899399 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:41.901123 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:41.902512 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:41.904341 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:41.905113 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:41.905313 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:41.910281 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:41.910290 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:41.910308 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:41.910348 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:41.913475 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:41.914216 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:41.914227 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:41.914235 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:41.914421 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:41.914819 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:41.914823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:41.914825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:41.915418 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:41.915727 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:41.916245 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 12:15:41.916700 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":385,"threads_started":1}
I20260811 12:15:41.920435 12330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c02285b11d0469c9274fd28141ab37c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41053 } }
I20260811 12:15:41.920830 12330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:41.920955 12330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c02285b11d0469c9274fd28141ab37c, State: Initialized, Role: FOLLOWER
I20260811 12:15:41.921589 12330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c02285b11d0469c9274fd28141ab37c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41053 } }
I20260811 12:15:41.921825 12330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:41.921949 12330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:41.922109 12330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:41.924605 12330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c02285b11d0469c9274fd28141ab37c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41053 } }
I20260811 12:15:41.925107 12330 leader_election.cc:302] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c02285b11d0469c9274fd28141ab37c; no voters:
I20260811 12:15:41.925837 12330 leader_election.cc:288] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:41.925928 12336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:41.927385 12336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [term 1 LEADER]: Becoming Leader. State: Replica: 6c02285b11d0469c9274fd28141ab37c, State: Running, Role: LEADER
I20260811 12:15:41.928107 12336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c02285b11d0469c9274fd28141ab37c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41053 } }
I20260811 12:15:41.928296 12330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:41.931172 12338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c02285b11d0469c9274fd28141ab37c. Latest consensus state: current_term: 1 leader_uuid: "6c02285b11d0469c9274fd28141ab37c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c02285b11d0469c9274fd28141ab37c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41053 } } }
I20260811 12:15:41.931399 12337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c02285b11d0469c9274fd28141ab37c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c02285b11d0469c9274fd28141ab37c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41053 } } }
I20260811 12:15:41.931718 12337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:41.931617 12338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:41.932613 12349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:41.938777 12349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:41.950239 12349 catalog_manager.cc:1347] Generated new cluster ID: 51a755b27ee640fea85517ddb3d57efc
I20260811 12:15:41.950368 12349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:41.960305 12349 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:41.961719 12349 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:41.968353 12349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c02285b11d0469c9274fd28141ab37c: Generated new TSK 0
I20260811 12:15:41.969432 12349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:15:42.034744 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41053
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:42.377102 12369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:42.377557 12369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:42.377731 12369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:42.387143 12369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:42.387275 12369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:42.387440 12369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:42.397794 12369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:42.399168 12369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:42.400529 12369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:42.413944 12383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:42.414047 12385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:42.414047 12387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:42.415067 12369 server_base.cc:1034] running on GCE node
I20260811 12:15:42.415673 12369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:42.416697 12369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:42.417958 12369 hybrid_clock.cc:648] HybridClock initialized: now 1786450542417887 us; error 58 us; skew 500 ppm
I20260811 12:15:42.418308 12369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:42 dist-test-slave-rmhg krb5kdc[12221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450542, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:42.421581 12369 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:15:42.423089 12369 webserver.cc:460] Webserver started at http://127.7.39.1:46747/ using document root <none> and password file <none>
I20260811 12:15:42.423801 12369 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:42.423923 12369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:42.424226 12369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:42.427282 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "44ff856ea99142cdaf8b39bf4320cead"
format_stamp: "Formatted at 2026-08-11 12:15:42 on dist-test-slave-rmhg"
server_key: "3ad5913fcc4372797b88aa3ee05562f6"
server_key_iv: "2b42f8ea8803eeea7f3e050241c05227"
server_key_version: "encryptionkey@0"
I20260811 12:15:42.428067 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "44ff856ea99142cdaf8b39bf4320cead"
format_stamp: "Formatted at 2026-08-11 12:15:42 on dist-test-slave-rmhg"
server_key: "3ad5913fcc4372797b88aa3ee05562f6"
server_key_iv: "2b42f8ea8803eeea7f3e050241c05227"
server_key_version: "encryptionkey@0"
I20260811 12:15:42.433548 12369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:15:42.437019 12396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:42.438400 12369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:42.438576 12369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "44ff856ea99142cdaf8b39bf4320cead"
format_stamp: "Formatted at 2026-08-11 12:15:42 on dist-test-slave-rmhg"
server_key: "3ad5913fcc4372797b88aa3ee05562f6"
server_key_iv: "2b42f8ea8803eeea7f3e050241c05227"
server_key_version: "encryptionkey@0"
I20260811 12:15:42.438798 12369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:42.471784 12369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:42.475948 12369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:42.476213 12369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:42.477268 12369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:42.478638 12369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:42.478722 12369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:42.478821 12369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:42.478864 12369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:42.510311 12369 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:33247
I20260811 12:15:42.510365 12573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:33247 every 8 connection(s)
I20260811 12:15:42.511644 12369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:42.517856 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 12369
I20260811 12:15:42.518064 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:42.518707 7324 external_mini_cluster.cc:1428] Setting key 10ffbb15e669585351a28014ca7f48dc
Aug 11 12:15:42 dist-test-slave-rmhg krb5kdc[12221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450542, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:42.533341 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:42.514297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:38993 (local address 127.7.39.62:41053)
0811 12:15:42.514717 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:42.514722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:42.515699 (+ 977us) server_negotiation.cc:407] Connection header received
0811 12:15:42.517573 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:42.517580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:42.517675 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:42.517779 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:42.519510 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:42.520379 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:42.522260 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:42.522864 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:42.523007 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:42.526922 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:42.526932 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:42.526937 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:42.526980 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:42.529893 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:42.530816 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:42.530821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:42.530824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:42.530956 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:42.531434 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:42.531438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:42.531440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:42.531730 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:42.531867 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:42.532687 (+ 820us) server_negotiation.cc:299] Negotiation successful
0811 12:15:42.532929 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":218,"threads_started":1}
I20260811 12:15:42.534446 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:42.514810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41053 (local address 127.7.39.1:38993)
0811 12:15:42.515521 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:42.515571 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:42.517275 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:42.518068 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:42.518077 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:42.518644 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:42.519298 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:42.519323 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:42.520596 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:42.520602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:42.522061 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:42.522075 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:42.523126 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:42.523131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:42.523351 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:42.524010 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:42.524039 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:42.526594 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:42.530068 (+ 3474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:42.530076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:42.530092 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:42.530663 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:42.531073 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:42.531077 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:42.531081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:42.531310 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:42.531865 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:42.531874 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:42.532281 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 12:15:42.532544 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 12:15:42.533131 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":292,"threads_started":1}
I20260811 12:15:42.536083 12574 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41053
I20260811 12:15:42.536437 12574 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:42.537456 12574 heartbeater.cc:511] Master 127.7.39.62:41053 requested a full tablet report, sending...
I20260811 12:15:42.539832 12289 ts_manager.cc:194] Registered new tserver with Master: 44ff856ea99142cdaf8b39bf4320cead (127.7.39.1:33247)
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
I20260811 12:15:42.541440 12289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:38993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:15:42.568930 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41053
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:42.905941 12583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:42.906262 12583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:42.906371 12583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:42.913872 12583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:42.914003 12583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:42.914151 12583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:42.924239 12583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:42.925666 12583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:42.927064 12583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:42.940668 12588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:42.940637 12595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:42.940601 12592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:42.941840 12583 server_base.cc:1034] running on GCE node
I20260811 12:15:42.942461 12583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:42.943487 12583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:42.944694 12583 hybrid_clock.cc:648] HybridClock initialized: now 1786450542944631 us; error 56 us; skew 500 ppm
I20260811 12:15:42.945029 12583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:42 dist-test-slave-rmhg krb5kdc[12221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450542, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:42.953394 12583 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:15:42.955021 12583 webserver.cc:460] Webserver started at http://127.7.39.2:40679/ using document root <none> and password file <none>
I20260811 12:15:42.955749 12583 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:42.955857 12583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:42.956156 12583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:42.959193 12583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "fcd410fb66654c31b6c19038a632e28a"
format_stamp: "Formatted at 2026-08-11 12:15:42 on dist-test-slave-rmhg"
server_key: "4f405dca96cadad4a89cd9d1bc73e691"
server_key_iv: "3602f45ac1d99d30e6ae8b224a4b6669"
server_key_version: "encryptionkey@0"
I20260811 12:15:42.959964 12583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "fcd410fb66654c31b6c19038a632e28a"
format_stamp: "Formatted at 2026-08-11 12:15:42 on dist-test-slave-rmhg"
server_key: "4f405dca96cadad4a89cd9d1bc73e691"
server_key_iv: "3602f45ac1d99d30e6ae8b224a4b6669"
server_key_version: "encryptionkey@0"
I20260811 12:15:42.965318 12583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:15:42.968823 12606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:42.970247 12583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:42.970402 12583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "fcd410fb66654c31b6c19038a632e28a"
format_stamp: "Formatted at 2026-08-11 12:15:42 on dist-test-slave-rmhg"
server_key: "4f405dca96cadad4a89cd9d1bc73e691"
server_key_iv: "3602f45ac1d99d30e6ae8b224a4b6669"
server_key_version: "encryptionkey@0"
I20260811 12:15:42.970598 12583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:42.995867 12583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:42.999996 12583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:43.000319 12583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:43.001382 12583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:43.002813 12583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:43.002895 12583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:43.003026 12583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:43.003089 12583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:43.034672 12583 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:37979
I20260811 12:15:43.034742 12779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:37979 every 8 connection(s)
I20260811 12:15:43.036000 12583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:43.042490 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 12583
I20260811 12:15:43.042699 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:43.043176 7324 external_mini_cluster.cc:1428] Setting key 656a77e0bce0f0fe82b6f3fb9659ccbb
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[12221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450542, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:43.057884 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:43.038632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:46979 (local address 127.7.39.62:41053)
0811 12:15:43.039078 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:43.039083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:43.040200 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 12:15:43.042182 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:43.042188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:43.042281 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:43.042382 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:43.044330 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.045252 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.046877 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.047520 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.047655 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:43.051774 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:43.051781 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:43.051785 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:43.051826 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:43.054675 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.055430 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.055435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.055439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.055569 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.056000 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.056003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.056005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.056303 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:43.056449 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:43.057201 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 12:15:43.057468 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":219,"threads_started":1}
I20260811 12:15:43.058885 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:43.039240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41053 (local address 127.7.39.2:46979)
0811 12:15:43.040017 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:43.040069 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:43.041827 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:43.042683 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:43.042695 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:43.043401 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:43.044161 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:43.044185 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.045414 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.045418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:43.046713 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:43.046727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.047733 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.047738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:43.047946 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:43.048607 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:43.048630 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:43.051504 (+ 2874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:43.054834 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:43.054844 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:43.054863 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:43.055306 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:43.055685 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:43.055690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:43.055694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:43.055890 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:43.056442 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:43.056450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:43.056810 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 12:15:43.057068 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 12:15:43.057685 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":281,"threads_started":1}
I20260811 12:15:43.060411 12780 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41053
I20260811 12:15:43.060760 12780 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:43.061725 12780 heartbeater.cc:511] Master 127.7.39.62:41053 requested a full tablet report, sending...
I20260811 12:15:43.063405 12289 ts_manager.cc:194] Registered new tserver with Master: fcd410fb66654c31b6c19038a632e28a (127.7.39.2:37979)
I20260811 12:15:43.064208 12289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:46979
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:15:43.096031 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41053
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:43.432190 12792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:43.432519 12792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:43.432659 12792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:43.440042 12792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:43.440178 12792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:43.440336 12792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:43.450500 12792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:43.451893 12792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:43.453328 12792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:43.466789 12803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:43.466812 12806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:43.467190 12802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:43.467971 12792 server_base.cc:1034] running on GCE node
I20260811 12:15:43.468575 12792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:43.469614 12792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:43.470813 12792 hybrid_clock.cc:648] HybridClock initialized: now 1786450543470765 us; error 46 us; skew 500 ppm
I20260811 12:15:43.471132 12792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[12221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450543, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:43.474426 12792 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:15:43.476064 12792 webserver.cc:460] Webserver started at http://127.7.39.3:43959/ using document root <none> and password file <none>
I20260811 12:15:43.476794 12792 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:43.476914 12792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:43.477262 12792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:43.480305 12792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "6f8d60b926a04e7893bbd8635c2f6647"
format_stamp: "Formatted at 2026-08-11 12:15:43 on dist-test-slave-rmhg"
server_key: "4fb51863d552f91fbf2c6f412241fb87"
server_key_iv: "7660ab4bfd589a706842062a16f365de"
server_key_version: "encryptionkey@0"
I20260811 12:15:43.481105 12792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "6f8d60b926a04e7893bbd8635c2f6647"
format_stamp: "Formatted at 2026-08-11 12:15:43 on dist-test-slave-rmhg"
server_key: "4fb51863d552f91fbf2c6f412241fb87"
server_key_iv: "7660ab4bfd589a706842062a16f365de"
server_key_version: "encryptionkey@0"
I20260811 12:15:43.486537 12792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:15:43.490366 12824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:43.492036 12792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 12:15:43.492220 12792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "6f8d60b926a04e7893bbd8635c2f6647"
format_stamp: "Formatted at 2026-08-11 12:15:43 on dist-test-slave-rmhg"
server_key: "4fb51863d552f91fbf2c6f412241fb87"
server_key_iv: "7660ab4bfd589a706842062a16f365de"
server_key_version: "encryptionkey@0"
I20260811 12:15:43.492427 12792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:43.506009 12792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:43.510520 12792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:43.510828 12792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:43.511914 12792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:43.513401 12792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:43.513486 12792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:43.513628 12792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:43.513692 12792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:43.544943 12574 heartbeater.cc:503] Master 127.7.39.62:41053 was elected leader, sending a full tablet report...
I20260811 12:15:43.545879 12792 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:46661
I20260811 12:15:43.545933 13005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:46661 every 8 connection(s)
I20260811 12:15:43.547297 12792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:43.548866 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 12792
I20260811 12:15:43.549032 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:43.549579 7324 external_mini_cluster.cc:1428] Setting key 659f3249ff78d3359506456b086bd1ad
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[12221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450543, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:43.567344 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:43.549855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:51821 (local address 127.7.39.62:41053)
0811 12:15:43.550081 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:43.550088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:43.551608 (+ 1520us) server_negotiation.cc:407] Connection header received
0811 12:15:43.553687 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:43.553691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:43.553761 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:43.553844 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:43.555273 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.556040 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.557686 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.558078 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.558185 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:43.561680 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:43.561685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:43.561688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:43.561722 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:43.564325 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.565221 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.565225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.565227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.565351 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.565761 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.565765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.565767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.566054 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:43.566185 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:43.566905 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 12:15:43.567098 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:15:43.568392 13010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:43.550466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41053 (local address 127.7.39.3:51821)
0811 12:15:43.551428 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:43.551469 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:43.553420 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:43.554023 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:43.554030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:43.554556 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:43.555140 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:43.555158 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.556179 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.556183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:43.557554 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:43.557566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.558253 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.558259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:43.558451 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:43.559072 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:43.559093 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:43.561436 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:43.564480 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:43.564490 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:43.564516 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:43.565052 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:43.565439 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:43.565442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:43.565444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:43.565649 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:43.566197 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:43.566204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:43.566554 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 12:15:43.566823 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 12:15:43.567250 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":342,"threads_started":1}
I20260811 12:15:43.569932 13006 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41053
I20260811 12:15:43.570261 13006 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:43.571022 13006 heartbeater.cc:511] Master 127.7.39.62:41053 requested a full tablet report, sending...
I20260811 12:15:43.572536 12289 ts_manager.cc:194] Registered new tserver with Master: 6f8d60b926a04e7893bbd8635c2f6647 (127.7.39.3:46661)
I20260811 12:15:43.573405 12289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:51821
I20260811 12:15:43.577713 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[12221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450543, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[12221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450543, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:43.636996 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:43.622300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.7.39.62:41053)
0811 12:15:43.622537 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:43.622544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:43.622918 (+ 374us) server_negotiation.cc:407] Connection header received
0811 12:15:43.623124 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:43.623128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:43.623196 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:43.623283 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:43.624715 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.625532 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.626877 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.627306 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.627424 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:43.631587 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:43.631593 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:43.631596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:43.631638 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:43.634265 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.634961 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.634965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.634968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.635113 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.635501 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.635505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.635507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.635860 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:43.636123 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:43.636415 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 12:15:43.636656 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 12:15:43.637663 12289 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40686
I20260811 12:15:43.639416 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12369
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[12221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450543, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:43.655642 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:43.640856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36636 (local address 127.7.39.1:33247)
0811 12:15:43.641495 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:43.641504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:43.641531 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:15:43.641589 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:43.641596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:43.641818 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:43.641945 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:43.643239 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.644277 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.645613 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.646226 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.646337 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:43.650272 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:43.650287 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:43.650303 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:43.650341 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:43.652954 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.653655 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.653661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.653664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.653791 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.654093 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.654095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.654097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.654442 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:43.654671 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:43.655010 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 12:15:43.655273 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":324,"threads_started":1}
W20260811 12:15:43.657054 12568 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36636
W20260811 12:15:43.658262 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12369: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:43.658463 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 12369
I20260811 12:15:43.689021 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12583
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[12221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450543, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:15:43.705564 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:43.690055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.7.39.2:37979)
0811 12:15:43.690583 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:43.690590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:43.690616 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:15:43.690678 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:43.690684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:43.690900 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:43.691065 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:43.692322 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.693485 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.694817 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.695394 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.695513 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:43.699778 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:43.699792 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:43.699812 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:43.699862 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:43.702686 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.703375 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.703383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.703385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.703519 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.703879 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.703883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.703885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.704257 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:43.704576 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:43.704889 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 12:15:43.705218 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":252,"threads_started":1}
W20260811 12:15:43.706943 12773 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33630
W20260811 12:15:43.708146 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12583: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:43.708300 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 12583
I20260811 12:15:43.731045 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12792
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[12221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450543, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:15:43.747960 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:43.732130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.7.39.3:46661)
0811 12:15:43.732802 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:43.732809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:43.732839 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:15:43.732900 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:43.732906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:43.733143 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:43.733324 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:43.734735 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.735801 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.737110 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:43.737757 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:43.737880 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:43.742346 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:43.742362 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:43.742379 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:43.742418 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:43.745004 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.745857 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.745864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.745867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.746002 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:43.746364 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:43.746368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:43.746370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:43.746731 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:43.747044 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:43.747347 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 12:15:43.747618 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":347,"threads_started":1}
W20260811 12:15:43.749383 12996 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51838
W20260811 12:15:43.750514 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12792: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:15:43.750690 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 12792
I20260811 12:15:43.774179 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12241
I20260811 12:15:43.775461 12325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:43.881731 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 12241
2026-08-11T12:15:43Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3898 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[13035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[13035](info): set up 2 sockets
Aug 11 12:15:43 dist-test-slave-rmhg krb5kdc[13035](info): commencing operation
krb5kdc: starting...
W20260811 12:15:45.127038 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:45 dist-test-slave-rmhg krb5kdc[13035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:45.372844 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:41307
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:41307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:45.716984 13052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:45.717386 13052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:45.717530 13052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:45.724978 13052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:45.725112 13052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:45.725198 13052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:45.725277 13052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:45.725337 13052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:45.735697 13052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:41307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:41307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:45.737624 13052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:45.739118 13052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:45.750873 13062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:45.750913 13064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:45.750945 13067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:45.751973 13052 server_base.cc:1034] running on GCE node
I20260811 12:15:45.752756 13052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:45.754259 13052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:45.755482 13052 hybrid_clock.cc:648] HybridClock initialized: now 1786450545755427 us; error 38 us; skew 500 ppm
I20260811 12:15:45.755885 13052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:45 dist-test-slave-rmhg krb5kdc[13035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450545, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:45.759093 13052 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:45.760672 13052 webserver.cc:460] Webserver started at http://127.7.39.62:40659/ using document root <none> and password file <none>
I20260811 12:15:45.761427 13052 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:45.761557 13052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:45.761873 13052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:45.765002 13052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b"
format_stamp: "Formatted at 2026-08-11 12:15:45 on dist-test-slave-rmhg"
server_key: "07dc69b0a0eec74aa53dde85d15a636d"
server_key_iv: "ae74a98ce4563dd8de449c4fbe65f606"
server_key_version: "encryptionkey@0"
I20260811 12:15:45.765897 13052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b"
format_stamp: "Formatted at 2026-08-11 12:15:45 on dist-test-slave-rmhg"
server_key: "07dc69b0a0eec74aa53dde85d15a636d"
server_key_iv: "ae74a98ce4563dd8de449c4fbe65f606"
server_key_version: "encryptionkey@0"
I20260811 12:15:45.771356 13052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:15:45.775174 13074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:45.776714 13052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:15:45.776891 13052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b"
format_stamp: "Formatted at 2026-08-11 12:15:45 on dist-test-slave-rmhg"
server_key: "07dc69b0a0eec74aa53dde85d15a636d"
server_key_iv: "ae74a98ce4563dd8de449c4fbe65f606"
server_key_version: "encryptionkey@0"
I20260811 12:15:45.777096 13052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:45.791684 13052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:45.795954 13052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:45.796247 13052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:45.815092 13052 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:41307
I20260811 12:15:45.815112 13163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:41307 every 8 connection(s)
I20260811 12:15:45.816633 13052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:45.820508 13165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:45.827131 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 13052
I20260811 12:15:45.827391 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:45.828258 7324 external_mini_cluster.cc:1428] Setting key 2df6439a8ac4ed608f17f4affb704947
I20260811 12:15:45.832408 13165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b: Bootstrap starting.
I20260811 12:15:45.835717 13165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:45.836870 13165 log.cc:824] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:45.840195 13165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b: No bootstrap required, opened a new log
Aug 11 12:15:45 dist-test-slave-rmhg krb5kdc[13035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:45.849545 13165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41307 } }
I20260811 12:15:45.850069 13165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:45.850196 13165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9431d9ebb24aaf9d46cd9fb23fa15b, State: Initialized, Role: FOLLOWER
I20260811 12:15:45.850819 13165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41307 } }
I20260811 12:15:45.851116 13165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:45.851269 13165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:45.851478 13165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:45.853240 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:45.830765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.7.39.62:41307)
0811 12:15:45.831630 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:45.831644 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:45.831689 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:15:45.833568 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:45.833594 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:45.834019 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:45.834411 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:45.835878 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:45.837480 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:45.839142 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:45.839843 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:45.839986 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:45.845075 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:45.845085 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:45.845103 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:45.845171 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:45.848411 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:45.849321 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:45.849328 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:45.849335 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:45.849552 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:45.850019 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:45.850025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:45.850028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:45.850831 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:45.851099 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:45.851631 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 12:15:45.852043 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":376,"threads_started":1}
I20260811 12:15:45.854115 13165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41307 } }
I20260811 12:15:45.854633 13165 leader_election.cc:302] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f9431d9ebb24aaf9d46cd9fb23fa15b; no voters:
I20260811 12:15:45.855276 13165 leader_election.cc:288] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:45.855441 13171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:45.856552 13171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [term 1 LEADER]: Becoming Leader. State: Replica: 4f9431d9ebb24aaf9d46cd9fb23fa15b, State: Running, Role: LEADER
I20260811 12:15:45.857203 13171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41307 } }
I20260811 12:15:45.857473 13165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:45.860414 13173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f9431d9ebb24aaf9d46cd9fb23fa15b. Latest consensus state: current_term: 1 leader_uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41307 } } }
I20260811 12:15:45.860881 13173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:45.861454 13172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9431d9ebb24aaf9d46cd9fb23fa15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41307 } } }
I20260811 12:15:45.861797 13172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:45.861907 13187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:45.867869 13187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:45.882249 13187 catalog_manager.cc:1347] Generated new cluster ID: cd69da83a7684e1f92121ee2de04240b
I20260811 12:15:45.882382 13187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:45.906198 13187 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:45.908241 13187 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:45.918100 13187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f9431d9ebb24aaf9d46cd9fb23fa15b: Generated new TSK 0
I20260811 12:15:45.919231 13187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:15:45.969965 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41307
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:46.314476 13205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:46.314805 13205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:46.314903 13205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:46.322338 13205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:46.322474 13205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:46.322592 13205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:46.332953 13205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:46.334367 13205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:46.335768 13205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:46.349344 13214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:46.349363 13216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:46.349328 13219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:46.350229 13205 server_base.cc:1034] running on GCE node
I20260811 12:15:46.350798 13205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:46.351719 13205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:46.352926 13205 hybrid_clock.cc:648] HybridClock initialized: now 1786450546352878 us; error 35 us; skew 500 ppm
I20260811 12:15:46.353291 13205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:46 dist-test-slave-rmhg krb5kdc[13035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450546, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:46.356436 13205 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:15:46.357981 13205 webserver.cc:460] Webserver started at http://127.7.39.1:41003/ using document root <none> and password file <none>
I20260811 12:15:46.358711 13205 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:46.358834 13205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:46.359141 13205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:46.362202 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "b86ea697057b41e5a8d8bfd826f2a436"
format_stamp: "Formatted at 2026-08-11 12:15:46 on dist-test-slave-rmhg"
server_key: "380d753a178f565c507e580f8b7ae479"
server_key_iv: "45b78262f1335d0588a814c9a696e014"
server_key_version: "encryptionkey@0"
I20260811 12:15:46.363023 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "b86ea697057b41e5a8d8bfd826f2a436"
format_stamp: "Formatted at 2026-08-11 12:15:46 on dist-test-slave-rmhg"
server_key: "380d753a178f565c507e580f8b7ae479"
server_key_iv: "45b78262f1335d0588a814c9a696e014"
server_key_version: "encryptionkey@0"
I20260811 12:15:46.368230 13205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:15:46.371824 13226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:46.373229 13205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:15:46.373399 13205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "b86ea697057b41e5a8d8bfd826f2a436"
format_stamp: "Formatted at 2026-08-11 12:15:46 on dist-test-slave-rmhg"
server_key: "380d753a178f565c507e580f8b7ae479"
server_key_iv: "45b78262f1335d0588a814c9a696e014"
server_key_version: "encryptionkey@0"
I20260811 12:15:46.373606 13205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:46.398433 13205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:46.402602 13205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:46.402928 13205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:46.403939 13205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:46.405346 13205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:46.405431 13205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:46.405556 13205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:46.405613 13205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:46.436995 13205 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:45163
I20260811 12:15:46.437069 13395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:45163 every 8 connection(s)
I20260811 12:15:46.438356 13205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:46.444011 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 13205
I20260811 12:15:46.444212 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:46.445017 7324 external_mini_cluster.cc:1428] Setting key 12275f103da57c767a547225a150ce53
Aug 11 12:15:46 dist-test-slave-rmhg krb5kdc[13035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450546, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:46.459642 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:46.440946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:55701 (local address 127.7.39.62:41307)
0811 12:15:46.441430 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:46.441434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:46.442374 (+ 940us) server_negotiation.cc:407] Connection header received
0811 12:15:46.444338 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:46.444345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:46.444462 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:46.444597 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:46.446429 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:46.447377 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:46.449011 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:46.449632 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:46.449770 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:46.453416 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:46.453423 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:46.453427 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:46.453479 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:46.456236 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:46.457243 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:46.457248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:46.457250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:46.457380 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:46.457864 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:46.457868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:46.457869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:46.458141 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:46.458282 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:46.459185 (+ 903us) server_negotiation.cc:299] Negotiation successful
0811 12:15:46.459372 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":270,"threads_started":1}
I20260811 12:15:46.461210 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:46.441457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41307 (local address 127.7.39.1:55701)
0811 12:15:46.442196 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:46.442246 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:46.443967 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:46.444845 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:46.444853 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:46.445456 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:46.446216 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:46.446250 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:46.447577 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:46.447581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:46.448853 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:46.448866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:46.449850 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:46.449855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:46.450064 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:46.450742 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:46.450765 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:46.453119 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:46.456399 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:46.456412 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:46.456432 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:46.457069 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:46.457482 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:46.457487 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:46.457490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:46.457762 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:46.458267 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:46.458276 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:46.458773 (+ 497us) client_negotiation.cc:769] Sending connection context
0811 12:15:46.459092 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 12:15:46.459733 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":276,"threads_started":1}
I20260811 12:15:46.462904 13400 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41307
I20260811 12:15:46.463229 13400 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:46.463970 13400 heartbeater.cc:511] Master 127.7.39.62:41307 requested a full tablet report, sending...
I20260811 12:15:46.466114 13104 ts_manager.cc:194] Registered new tserver with Master: b86ea697057b41e5a8d8bfd826f2a436 (127.7.39.1:45163)
I20260811 12:15:46.467602 13104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:55701
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:15:46.497671 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41307
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:46.834995 13412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:46.835318 13412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:46.835443 13412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:46.842751 13412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:46.842893 13412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:46.843061 13412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:46.852860 13412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:46.854257 13412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:46.855657 13412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:46.868911 13427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:46.869036 13419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:46.868903 13418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:46.870110 13412 server_base.cc:1034] running on GCE node
I20260811 12:15:46.870769 13412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:46.871757 13412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:46.872977 13412 hybrid_clock.cc:648] HybridClock initialized: now 1786450546872925 us; error 39 us; skew 500 ppm
I20260811 12:15:46.873370 13412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:46 dist-test-slave-rmhg krb5kdc[13035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450546, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:46.876866 13412 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:15:46.878583 13412 webserver.cc:460] Webserver started at http://127.7.39.2:41139/ using document root <none> and password file <none>
I20260811 12:15:46.879351 13412 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:46.879488 13412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:46.879818 13412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:46.883019 13412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "b95fbd52294c40589f1e72faae3327dd"
format_stamp: "Formatted at 2026-08-11 12:15:46 on dist-test-slave-rmhg"
server_key: "634dff04e76400610e87e544c6fcb0a8"
server_key_iv: "a174ca0d3d97f43733210ffd647a9c00"
server_key_version: "encryptionkey@0"
I20260811 12:15:46.883862 13412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "b95fbd52294c40589f1e72faae3327dd"
format_stamp: "Formatted at 2026-08-11 12:15:46 on dist-test-slave-rmhg"
server_key: "634dff04e76400610e87e544c6fcb0a8"
server_key_iv: "a174ca0d3d97f43733210ffd647a9c00"
server_key_version: "encryptionkey@0"
I20260811 12:15:46.889546 13412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:15:46.893219 13434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:46.894716 13412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:46.894897 13412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "b95fbd52294c40589f1e72faae3327dd"
format_stamp: "Formatted at 2026-08-11 12:15:46 on dist-test-slave-rmhg"
server_key: "634dff04e76400610e87e544c6fcb0a8"
server_key_iv: "a174ca0d3d97f43733210ffd647a9c00"
server_key_version: "encryptionkey@0"
I20260811 12:15:46.895114 13412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:46.916239 13412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:46.920985 13412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:46.921334 13412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:46.922364 13412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:46.923796 13412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:46.923883 13412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:46.924005 13412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:46.924057 13412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:46.954658 13412 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:39615
I20260811 12:15:46.954722 13612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:39615 every 8 connection(s)
I20260811 12:15:46.956065 13412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:46.960696 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 13412
I20260811 12:15:46.960894 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:46.961489 7324 external_mini_cluster.cc:1428] Setting key 4967d52ecd4e2a4b24adcf6eecd69a82
Aug 11 12:15:46 dist-test-slave-rmhg krb5kdc[13035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450546, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:46.978407 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:46.958710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:53897 (local address 127.7.39.62:41307)
0811 12:15:46.958893 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:46.958898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:46.960090 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 12:15:46.962552 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:46.962557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:46.962646 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:46.962740 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:46.964544 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:46.965306 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:46.967416 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:46.967862 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:46.967956 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:46.971996 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:46.972003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:46.972006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:46.972043 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:46.975251 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:46.975997 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:46.976003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:46.976007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:46.976172 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:46.976555 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:46.976559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:46.976563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:46.976929 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:46.977073 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:46.977841 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 12:15:46.978080 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:15:46.979470 13617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:46.959186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41307 (local address 127.7.39.2:53897)
0811 12:15:46.959923 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:46.959966 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:46.962281 (+ 2315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:46.963001 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:46.963010 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:46.963533 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:46.964335 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:46.964361 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:46.965481 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:46.965486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:46.967262 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:46.967277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:46.968078 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:46.968084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:46.968355 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:46.969064 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:46.969095 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:46.971729 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:46.975390 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:46.975398 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:46.975413 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:46.975860 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:46.976271 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:46.976275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:46.976277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:46.976456 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:46.977035 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:46.977041 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:46.977456 (+ 415us) client_negotiation.cc:769] Sending connection context
0811 12:15:46.977703 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 12:15:46.978134 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":274,"threads_started":1}
I20260811 12:15:46.981232 13614 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41307
I20260811 12:15:46.981554 13614 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:46.982380 13614 heartbeater.cc:511] Master 127.7.39.62:41307 requested a full tablet report, sending...
I20260811 12:15:46.983875 13104 ts_manager.cc:194] Registered new tserver with Master: b95fbd52294c40589f1e72faae3327dd (127.7.39.2:39615)
I20260811 12:15:46.984782 13104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:53897
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:15:47.014456 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41307
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:47.355084 13624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:47.355413 13624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:47.355540 13624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:47.363126 13624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:47.363278 13624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:47.363436 13624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:47.373708 13624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:47.375180 13624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:47.376603 13624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:47.390748 13634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:47.390782 13632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:47.390698 13637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:47.391844 13624 server_base.cc:1034] running on GCE node
I20260811 12:15:47.392484 13624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:47.393539 13624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:47.394742 13624 hybrid_clock.cc:648] HybridClock initialized: now 1786450547394685 us; error 45 us; skew 500 ppm
I20260811 12:15:47.395066 13624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:47 dist-test-slave-rmhg krb5kdc[13035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450547, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:47.398288 13624 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:15:47.399891 13624 webserver.cc:460] Webserver started at http://127.7.39.3:44237/ using document root <none> and password file <none>
I20260811 12:15:47.400665 13624 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:47.400781 13624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:47.401069 13624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:47.404182 13624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "cce9cf8b34114327a73103a7313fa4b3"
format_stamp: "Formatted at 2026-08-11 12:15:47 on dist-test-slave-rmhg"
server_key: "10c8db9c1e60b27609496d38f01bbbc9"
server_key_iv: "c35ff3fad8d959e43b67435bb456af15"
server_key_version: "encryptionkey@0"
I20260811 12:15:47.404986 13624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "cce9cf8b34114327a73103a7313fa4b3"
format_stamp: "Formatted at 2026-08-11 12:15:47 on dist-test-slave-rmhg"
server_key: "10c8db9c1e60b27609496d38f01bbbc9"
server_key_iv: "c35ff3fad8d959e43b67435bb456af15"
server_key_version: "encryptionkey@0"
I20260811 12:15:47.410435 13624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:15:47.414206 13650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:47.415630 13624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:15:47.415777 13624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "cce9cf8b34114327a73103a7313fa4b3"
format_stamp: "Formatted at 2026-08-11 12:15:47 on dist-test-slave-rmhg"
server_key: "10c8db9c1e60b27609496d38f01bbbc9"
server_key_iv: "c35ff3fad8d959e43b67435bb456af15"
server_key_version: "encryptionkey@0"
I20260811 12:15:47.415961 13624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:47.443974 13624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:47.448354 13624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:47.448626 13624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:47.449780 13624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:47.451175 13624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:47.451256 13624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:47.451340 13624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:47.451385 13624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:47.470847 13400 heartbeater.cc:503] Master 127.7.39.62:41307 was elected leader, sending a full tablet report...
I20260811 12:15:47.483619 13624 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:33367
I20260811 12:15:47.483693 13816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:33367 every 8 connection(s)
I20260811 12:15:47.484964 13624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:47.488682 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 13624
I20260811 12:15:47.488893 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:47.489459 7324 external_mini_cluster.cc:1428] Setting key 3ae2f1b6344a985c23634712da3191e3
Aug 11 12:15:47 dist-test-slave-rmhg krb5kdc[13035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450547, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:47.506749 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:47.487680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:60759 (local address 127.7.39.62:41307)
0811 12:15:47.488148 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:47.488153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:47.489317 (+ 1164us) server_negotiation.cc:407] Connection header received
0811 12:15:47.491448 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:47.491454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:47.491549 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:47.491669 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:47.493778 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.494642 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.496283 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.496720 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.496820 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:47.500505 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:47.500511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:47.500514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:47.500555 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:47.503301 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.504175 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.504181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.504184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.504334 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.504885 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.504889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.504891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.505322 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:47.505481 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:47.506249 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 12:15:47.506446 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":239,"threads_started":1}
I20260811 12:15:47.507831 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:47.488264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41307 (local address 127.7.39.3:60759)
0811 12:15:47.489107 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:47.489197 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:47.491142 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:47.491997 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:47.492011 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:47.492848 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:47.493569 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:47.493591 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.494762 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.494766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:47.496124 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:47.496138 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.496918 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.496923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:47.497188 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:47.497853 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:47.497877 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:47.500255 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:47.503471 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:47.503479 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:47.503496 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:47.504035 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:47.504466 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:47.504469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:47.504472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:47.504744 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:47.505469 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:47.505476 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:47.505886 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 12:15:47.506129 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 12:15:47.506576 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":316,"threads_started":1}
I20260811 12:15:47.509476 13817 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41307
I20260811 12:15:47.509835 13817 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:47.510607 13817 heartbeater.cc:511] Master 127.7.39.62:41307 requested a full tablet report, sending...
I20260811 12:15:47.512212 13104 ts_manager.cc:194] Registered new tserver with Master: cce9cf8b34114327a73103a7313fa4b3 (127.7.39.3:33367)
I20260811 12:15:47.513068 13104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:60759
I20260811 12:15:47.518010 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:15:47 dist-test-slave-rmhg krb5kdc[13035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:15:47 dist-test-slave-rmhg krb5kdc[13035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:47.577704 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:47.561226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.7.39.1:45163)
0811 12:15:47.562035 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:47.562042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:47.562071 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:15:47.562136 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:47.562142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:47.562365 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:47.562524 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:47.563895 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.564969 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.566327 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.567027 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.567147 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:47.571251 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:47.571267 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:47.571285 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:47.571328 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:47.574649 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.575460 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.575505 (+ 45us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.575507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.575673 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.576045 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.576049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.576051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.576443 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:47.576756 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:47.576990 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 12:15:47.577288 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":481,"threads_started":1}
Aug 11 12:15:47 dist-test-slave-rmhg krb5kdc[13035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:47.596623 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:47.582399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.7.39.62:41307)
0811 12:15:47.582616 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:47.582622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:47.582865 (+ 243us) server_negotiation.cc:407] Connection header received
0811 12:15:47.583040 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:47.583045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:47.583117 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:47.583204 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:47.584569 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.585382 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.586727 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.587173 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.587292 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:47.591603 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:47.591610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:47.591613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:47.591663 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:47.594166 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.594870 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.594875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.594878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.595038 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.595341 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.595344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.595346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.595701 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:47.595899 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:47.596152 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 12:15:47.596338 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 12:15:47.597322 13104 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56130
I20260811 12:15:47.598476 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 13205
I20260811 12:15:47.612212 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:47.599448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.7.39.1:45163)
0811 12:15:47.599653 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:47.599659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:47.599967 (+ 308us) server_negotiation.cc:407] Connection header received
0811 12:15:47.600149 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:47.600153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:47.600253 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:47.600343 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:47.601670 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.602373 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.603655 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.604110 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.604232 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:47.606995 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:47.607001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:47.607004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:47.607044 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:47.609465 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.610169 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.610174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.610177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.610314 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.610704 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.610708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.610710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.610996 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:47.611229 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:47.611624 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 12:15:47.611859 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:15:47.612967 13391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:47.697420 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 13205
I20260811 12:15:47.718716 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 13412
Aug 11 12:15:47 dist-test-slave-rmhg krb5kdc[13035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:15:47.735080 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:47.719786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.7.39.2:39615)
0811 12:15:47.720471 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:47.720479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:47.720507 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:15:47.720568 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:47.720574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:47.720790 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:47.720924 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:47.722146 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.723191 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.724444 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.725026 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.725132 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:47.729488 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:47.729502 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:47.729519 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:47.729566 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:47.732369 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.733049 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.733056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.733058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.733251 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.733598 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.733600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.733603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.733969 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:47.734227 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:47.734474 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 12:15:47.734703 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":404,"threads_started":1}
I20260811 12:15:47.736560 13607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:47.822391 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 13412
I20260811 12:15:47.841550 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 13624
Aug 11 12:15:47 dist-test-slave-rmhg krb5kdc[13035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:15:47.858743 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:47.842618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.7.39.3:33367)
0811 12:15:47.843298 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:47.843311 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:47.843343 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:15:47.843405 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:47.843411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:47.843639 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:47.843789 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:47.845247 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.846328 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.847680 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:47.848270 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:47.848391 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:47.852857 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:47.852871 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:47.852888 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:47.852925 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:47.855754 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.856478 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.856486 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.856489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.856635 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:47.857008 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:47.857012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:47.857014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:47.857430 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:47.857754 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:47.858102 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 12:15:47.858393 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":356,"threads_started":1}
I20260811 12:15:47.860193 13809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:47.942795 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 13624
I20260811 12:15:47.963805 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 13052
I20260811 12:15:47.965034 13156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:48.070070 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 13052
2026-08-11T12:15:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4190 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:48 dist-test-slave-rmhg krb5kdc[13848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:48 dist-test-slave-rmhg krb5kdc[13848](info): set up 2 sockets
Aug 11 12:15:48 dist-test-slave-rmhg krb5kdc[13848](info): commencing operation
krb5kdc: starting...
W20260811 12:15:49.313571 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:15:49 dist-test-slave-rmhg krb5kdc[13848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:15:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:15:49.557348 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:46373
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:35057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:46373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:49.898092 13868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:49.898722 13868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:49.898937 13868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:49.908435 13868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:49.908577 13868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:49.908632 13868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:49.908689 13868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:49.908738 13868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:49.919304 13868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:46373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:46373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:49.920704 13868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:49.922160 13868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:49.934849 13876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:49.934911 13878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:49.934823 13875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:49.938721 13868 server_base.cc:1034] running on GCE node
I20260811 12:15:49.939625 13868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:49.941339 13868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:49.942572 13868 hybrid_clock.cc:648] HybridClock initialized: now 1786450549942507 us; error 66 us; skew 500 ppm
I20260811 12:15:49.942955 13868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:49 dist-test-slave-rmhg krb5kdc[13848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450549, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:49.946229 13868 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:15:49.947539 13868 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:49.947654 13868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:49.948005 13868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:49.951249 13868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "38d7071dfbad4bd8ad3ca8d6640a7360"
format_stamp: "Formatted at 2026-08-11 12:15:49 on dist-test-slave-rmhg"
server_key: "c952fb947963353a05c3152aed88335b"
server_key_iv: "93eb51b331a52a1f337d8483f3feaf3d"
server_key_version: "encryptionkey@0"
I20260811 12:15:49.952076 13868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "38d7071dfbad4bd8ad3ca8d6640a7360"
format_stamp: "Formatted at 2026-08-11 12:15:49 on dist-test-slave-rmhg"
server_key: "c952fb947963353a05c3152aed88335b"
server_key_iv: "93eb51b331a52a1f337d8483f3feaf3d"
server_key_version: "encryptionkey@0"
I20260811 12:15:49.957504 13868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 12:15:49.961217 13887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:49.962728 13868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:49.962929 13868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "38d7071dfbad4bd8ad3ca8d6640a7360"
format_stamp: "Formatted at 2026-08-11 12:15:49 on dist-test-slave-rmhg"
server_key: "c952fb947963353a05c3152aed88335b"
server_key_iv: "93eb51b331a52a1f337d8483f3feaf3d"
server_key_version: "encryptionkey@0"
I20260811 12:15:49.963158 13868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:49.982151 13868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:49.986742 13868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:49.986997 13868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:50.004838 13868 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:46373
I20260811 12:15:50.004851 13971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:46373 every 8 connection(s)
I20260811 12:15:50.005863 13868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:50.009752 13972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:50.012248 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 13868
I20260811 12:15:50.012471 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:50.013109 7324 external_mini_cluster.cc:1428] Setting key e378d1be53491f102fe93f00c7a21971
I20260811 12:15:50.022012 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360: Bootstrap starting.
I20260811 12:15:50.026033 13972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:50.027395 13972 log.cc:824] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:15:50 dist-test-slave-rmhg krb5kdc[13848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:50.030483 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360: No bootstrap required, opened a new log
I20260811 12:15:50.037024 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:50.015226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54966 (local address 127.7.39.62:46373)
0811 12:15:50.016095 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:50.016110 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:50.016153 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:15:50.017846 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:50.017871 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:50.018308 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:50.018656 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:50.020578 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:50.021975 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:50.023889 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:50.024661 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:50.024803 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:50.029581 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:50.029589 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:50.029606 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:50.029643 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:50.032732 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:50.033509 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:50.033517 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:50.033523 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:50.033709 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:50.034094 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:50.034097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:50.034099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:50.034706 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:50.035016 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:50.035484 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 12:15:50.035949 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":387,"threads_started":1}
I20260811 12:15:50.038240 13972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7071dfbad4bd8ad3ca8d6640a7360" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46373 } }
I20260811 12:15:50.038625 13972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:50.038728 13972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d7071dfbad4bd8ad3ca8d6640a7360, State: Initialized, Role: FOLLOWER
I20260811 12:15:50.039273 13972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7071dfbad4bd8ad3ca8d6640a7360" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46373 } }
I20260811 12:15:50.039505 13972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:50.039640 13972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:50.039813 13972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:50.042380 13972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7071dfbad4bd8ad3ca8d6640a7360" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46373 } }
I20260811 12:15:50.042858 13972 leader_election.cc:302] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38d7071dfbad4bd8ad3ca8d6640a7360; no voters:
I20260811 12:15:50.043358 13972 leader_election.cc:288] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:50.043505 13978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:50.044219 13978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [term 1 LEADER]: Becoming Leader. State: Replica: 38d7071dfbad4bd8ad3ca8d6640a7360, State: Running, Role: LEADER
I20260811 12:15:50.044742 13978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7071dfbad4bd8ad3ca8d6640a7360" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46373 } }
I20260811 12:15:50.045423 13972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:50.047587 13981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38d7071dfbad4bd8ad3ca8d6640a7360. Latest consensus state: current_term: 1 leader_uuid: "38d7071dfbad4bd8ad3ca8d6640a7360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7071dfbad4bd8ad3ca8d6640a7360" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46373 } } }
I20260811 12:15:50.047772 13980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38d7071dfbad4bd8ad3ca8d6640a7360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7071dfbad4bd8ad3ca8d6640a7360" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46373 } } }
I20260811 12:15:50.047998 13981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:50.048051 13980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:50.048952 13994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:50.054359 13994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:50.058835 14009 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:15:50.063640 13994 catalog_manager.cc:1347] Generated new cluster ID: 3abda19523624acf8db1ed431291f527
I20260811 12:15:50.063766 13994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:50.116730 13994 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:50.118851 13994 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:50.130800 13994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360: Generated new TSK 0
I20260811 12:15:50.131983 13994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:15:50.205513 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:46373
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:35057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:50.553107 14015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:50.553470 14015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:50.553633 14015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:50.560930 14015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:50.561101 14015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:50.561308 14015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:50.571475 14015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:50.572831 14015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:50.574276 14015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:50.587414 14022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:50.587459 14026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:50.587489 14023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:50.588357 14015 server_base.cc:1034] running on GCE node
I20260811 12:15:50.588966 14015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:50.590015 14015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:50.591224 14015 hybrid_clock.cc:648] HybridClock initialized: now 1786450550591160 us; error 60 us; skew 500 ppm
I20260811 12:15:50.591665 14015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:50 dist-test-slave-rmhg krb5kdc[13848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450550, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:50.594872 14015 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:15:50.596056 14015 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:50.596153 14015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:50.596453 14015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:50.599517 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "dd49096037a04f06aaf07d38b4356b57"
format_stamp: "Formatted at 2026-08-11 12:15:50 on dist-test-slave-rmhg"
server_key: "528a68fab1ac11453d63c6dafd201bff"
server_key_iv: "4fa95acea07b16792c3513d90e98cd08"
server_key_version: "encryptionkey@0"
I20260811 12:15:50.600338 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "dd49096037a04f06aaf07d38b4356b57"
format_stamp: "Formatted at 2026-08-11 12:15:50 on dist-test-slave-rmhg"
server_key: "528a68fab1ac11453d63c6dafd201bff"
server_key_iv: "4fa95acea07b16792c3513d90e98cd08"
server_key_version: "encryptionkey@0"
I20260811 12:15:50.605650 14015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:15:50.609328 14038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:50.610697 14015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:50.610844 14015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "dd49096037a04f06aaf07d38b4356b57"
format_stamp: "Formatted at 2026-08-11 12:15:50 on dist-test-slave-rmhg"
server_key: "528a68fab1ac11453d63c6dafd201bff"
server_key_iv: "4fa95acea07b16792c3513d90e98cd08"
server_key_version: "encryptionkey@0"
I20260811 12:15:50.611037 14015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:50.625622 14015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:50.629683 14015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:50.629951 14015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:50.631054 14015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:50.632431 14015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:50.632524 14015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:50.632627 14015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:50.632709 14015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:50.663816 14015 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:40899
I20260811 12:15:50.663870 14226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:40899 every 8 connection(s)
I20260811 12:15:50.664775 14015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:50.665315 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 14015
I20260811 12:15:50.665555 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:50.666405 7324 external_mini_cluster.cc:1428] Setting key 78a042d09b863b6f1749ecf0d70a31d5
Aug 11 12:15:50 dist-test-slave-rmhg krb5kdc[13848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450550, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:50.687047 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:50.667729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:59047 (local address 127.7.39.62:46373)
0811 12:15:50.668175 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:50.668180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:50.669420 (+ 1240us) server_negotiation.cc:407] Connection header received
0811 12:15:50.671227 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:50.671233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:50.671323 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:50.671420 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:50.672967 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:50.673783 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:50.675670 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:50.676222 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:50.676354 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:50.680188 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:50.680194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:50.680199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:50.680242 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:50.683489 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:50.684383 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:50.684389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:50.684392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:50.684568 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:50.685083 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:50.685088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:50.685091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:50.685490 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:50.685646 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:50.686421 (+ 775us) server_negotiation.cc:299] Negotiation successful
0811 12:15:50.686669 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":238,"threads_started":1}
I20260811 12:15:50.687994 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:50.668463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:46373 (local address 127.7.39.1:59047)
0811 12:15:50.669280 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:50.669323 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:50.670956 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:50.671685 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:50.671693 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:50.672229 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:50.672796 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:50.672814 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:50.673935 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:50.673939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:50.675489 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:50.675504 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:50.676436 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:50.676441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:50.676691 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:50.677483 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:50.677513 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:50.679906 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:50.683694 (+ 3788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:50.683703 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:50.683720 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:50.684224 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:50.684713 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:50.684718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:50.684720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:50.684961 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:50.685624 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:50.685632 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:50.686014 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 12:15:50.686274 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 12:15:50.686740 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":320,"threads_started":1}
I20260811 12:15:50.689745 14227 heartbeater.cc:347] Connected to a master server at 127.7.39.62:46373
I20260811 12:15:50.690083 14227 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:50.690790 14227 heartbeater.cc:511] Master 127.7.39.62:46373 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
I20260811 12:15:50.693135 13914 ts_manager.cc:194] Registered new tserver with Master: dd49096037a04f06aaf07d38b4356b57 (127.7.39.1:40899)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
I20260811 12:15:50.694798 13914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:59047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:15:50.722357 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:46373
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:35057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:51.064343 14236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:51.064678 14236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:51.064817 14236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:51.072324 14236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:51.072453 14236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:51.072623 14236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:51.082818 14236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:51.084290 14236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:51.085798 14236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:51.099303 14241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:51.099225 14246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:51.099282 14248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:51.100659 14236 server_base.cc:1034] running on GCE node
I20260811 12:15:51.101364 14236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:51.102363 14236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:51.103578 14236 hybrid_clock.cc:648] HybridClock initialized: now 1786450551103522 us; error 47 us; skew 500 ppm
I20260811 12:15:51.103880 14236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:51 dist-test-slave-rmhg krb5kdc[13848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:51.107227 14236 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:15:51.108374 14236 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:51.108557 14236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:51.108898 14236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:51.112099 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "6afc2f76e6dd49bf9dcc1336094b2c24"
format_stamp: "Formatted at 2026-08-11 12:15:51 on dist-test-slave-rmhg"
server_key: "9edab8a61addbf2e248cf95694c79a73"
server_key_iv: "db73ba4a1676afe9624e0af115300402"
server_key_version: "encryptionkey@0"
I20260811 12:15:51.113131 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "6afc2f76e6dd49bf9dcc1336094b2c24"
format_stamp: "Formatted at 2026-08-11 12:15:51 on dist-test-slave-rmhg"
server_key: "9edab8a61addbf2e248cf95694c79a73"
server_key_iv: "db73ba4a1676afe9624e0af115300402"
server_key_version: "encryptionkey@0"
I20260811 12:15:51.119069 14236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:15:51.123059 14258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:51.124631 14236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:15:51.124811 14236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "6afc2f76e6dd49bf9dcc1336094b2c24"
format_stamp: "Formatted at 2026-08-11 12:15:51 on dist-test-slave-rmhg"
server_key: "9edab8a61addbf2e248cf95694c79a73"
server_key_iv: "db73ba4a1676afe9624e0af115300402"
server_key_version: "encryptionkey@0"
I20260811 12:15:51.125046 14236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:51.160038 14236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:51.164618 14236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:51.164898 14236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:51.165977 14236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:51.167520 14236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:51.167637 14236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:51.167753 14236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:51.167824 14236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:51.198843 14236 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:36521
I20260811 12:15:51.198891 14427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:36521 every 8 connection(s)
I20260811 12:15:51.199717 14236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:51.207304 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 14236
I20260811 12:15:51.207506 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:51.207935 7324 external_mini_cluster.cc:1428] Setting key b4f0928c30f795040ea6d37cbeedb059
Aug 11 12:15:51 dist-test-slave-rmhg krb5kdc[13848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:51.220235 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.202350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:33173 (local address 127.7.39.62:46373)
0811 12:15:51.202823 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.202828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.203724 (+ 896us) server_negotiation.cc:407] Connection header received
0811 12:15:51.205631 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.205638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.205741 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.205843 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:51.207448 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.208239 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.210139 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.210581 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.210705 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.214150 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:51.214155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:51.214158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:51.214195 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:51.216897 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:51.217722 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:51.217726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:51.217728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:51.217864 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:51.218351 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:51.218355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:51.218357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:51.218633 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:51.218779 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:51.219597 (+ 818us) server_negotiation.cc:299] Negotiation successful
0811 12:15:51.219850 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":240,"threads_started":1}
I20260811 12:15:51.221498 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.202794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:46373 (local address 127.7.39.2:33173)
0811 12:15:51.203539 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:51.203588 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:51.205332 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:51.206091 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:51.206100 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:51.206654 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:51.207257 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.207277 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.208403 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.208408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.209999 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.210013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.210817 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.210824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.211044 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.211698 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:51.211721 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:51.213914 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:51.217045 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:51.217055 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:51.217073 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:51.217582 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:51.217976 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:51.217981 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:51.217985 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:51.218231 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:51.218779 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:51.218788 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:51.219155 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 12:15:51.219463 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 12:15:51.220049 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":293,"threads_started":1}
I20260811 12:15:51.223034 14429 heartbeater.cc:347] Connected to a master server at 127.7.39.62:46373
I20260811 12:15:51.223337 14429 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:51.224010 14429 heartbeater.cc:511] Master 127.7.39.62:46373 requested a full tablet report, sending...
I20260811 12:15:51.225586 13914 ts_manager.cc:194] Registered new tserver with Master: 6afc2f76e6dd49bf9dcc1336094b2c24 (127.7.39.2:36521)
I20260811 12:15:51.226773 13914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:33173
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:15:51.258803 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:46373
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:35057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:15:51.598292 14443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:51.598618 14443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:51.598760 14443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:51.606330 14443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:51.606463 14443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:51.606621 14443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:51.616703 14443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:51.618134 14443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:51.619505 14443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:51.633101 14452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:51.633319 14453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:51.634002 14459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:51.634708 14443 server_base.cc:1034] running on GCE node
I20260811 12:15:51.635318 14443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:51.636281 14443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:51.637495 14443 hybrid_clock.cc:648] HybridClock initialized: now 1786450551637441 us; error 45 us; skew 500 ppm
I20260811 12:15:51.637830 14443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:15:51 dist-test-slave-rmhg krb5kdc[13848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:15:51.640933 14443 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:15:51.642163 14443 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:51.642271 14443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:51.642596 14443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:51.645750 14443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "596f87909a90403cbb2d659dbc42249e"
format_stamp: "Formatted at 2026-08-11 12:15:51 on dist-test-slave-rmhg"
server_key: "8e5218ab940ad43848e8a185027c5b6c"
server_key_iv: "c8285e4eba00a550dd4e912d7dd2a889"
server_key_version: "encryptionkey@0"
I20260811 12:15:51.646596 14443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "596f87909a90403cbb2d659dbc42249e"
format_stamp: "Formatted at 2026-08-11 12:15:51 on dist-test-slave-rmhg"
server_key: "8e5218ab940ad43848e8a185027c5b6c"
server_key_iv: "c8285e4eba00a550dd4e912d7dd2a889"
server_key_version: "encryptionkey@0"
I20260811 12:15:51.651752 14443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:15:51.655355 14471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:51.656662 14443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:15:51.656806 14443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "596f87909a90403cbb2d659dbc42249e"
format_stamp: "Formatted at 2026-08-11 12:15:51 on dist-test-slave-rmhg"
server_key: "8e5218ab940ad43848e8a185027c5b6c"
server_key_iv: "c8285e4eba00a550dd4e912d7dd2a889"
server_key_version: "encryptionkey@0"
I20260811 12:15:51.657003 14443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:51.681830 14443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:51.685761 14443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:51.686033 14443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:51.687216 14443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:51.688575 14443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:51.688813 14443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:51.688936 14443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:51.689015 14443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:51.698078 14227 heartbeater.cc:503] Master 127.7.39.62:46373 was elected leader, sending a full tablet report...
I20260811 12:15:51.720594 14443 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:43251
I20260811 12:15:51.720636 14635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:43251 every 8 connection(s)
I20260811 12:15:51.721549 14443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:51.722529 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 14443
I20260811 12:15:51.722738 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:51.723274 7324 external_mini_cluster.cc:1428] Setting key a4783281be20fe1262c28baf28567146
Aug 11 12:15:51 dist-test-slave-rmhg krb5kdc[13848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:15:51.742244 14638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.724296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:56995 (local address 127.7.39.62:46373)
0811 12:15:51.724908 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.724914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.726045 (+ 1131us) server_negotiation.cc:407] Connection header received
0811 12:15:51.728036 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.728040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.728141 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.728248 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:51.729901 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.730770 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.732354 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.732802 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.732900 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.736327 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:51.736332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:51.736335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:51.736369 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:51.739393 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:51.740119 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:51.740123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:51.740125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:51.740249 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:51.740608 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:51.740611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:51.740613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:51.740876 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:51.740985 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:51.741709 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 12:15:51.741888 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":244,"threads_started":1}
I20260811 12:15:51.743219 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.725000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:46373 (local address 127.7.39.3:56995)
0811 12:15:51.725850 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:51.725921 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:51.727748 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:51.728439 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:51.728447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:51.729035 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:51.729733 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.729752 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.730898 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.730902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.732181 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.732193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.733009 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.733014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.733261 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.733914 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:15:51.733936 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:51.736098 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:51.739510 (+ 3412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:51.739518 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:51.739535 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:51.740002 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:51.740329 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:15:51.740332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:15:51.740334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:15:51.740522 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:15:51.740966 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:51.740972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:51.741383 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 12:15:51.741609 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 12:15:51.742061 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":337,"threads_started":1}
I20260811 12:15:51.744635 14636 heartbeater.cc:347] Connected to a master server at 127.7.39.62:46373
I20260811 12:15:51.744947 14636 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:51.745649 14636 heartbeater.cc:511] Master 127.7.39.62:46373 requested a full tablet report, sending...
I20260811 12:15:51.747123 13914 ts_manager.cc:194] Registered new tserver with Master: 596f87909a90403cbb2d659dbc42249e (127.7.39.3:43251)
I20260811 12:15:51.747952 13914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:56995
I20260811 12:15:51.751713 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:15:51.768777 14638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.756501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54980 (local address 127.7.39.62:46373)
0811 12:15:51.756695 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.756700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.757276 (+ 576us) server_negotiation.cc:407] Connection header received
0811 12:15:51.757415 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.757420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.757487 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.757577 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:51.758982 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.759719 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.760969 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.761524 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.761615 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.763801 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:51.763806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:51.763810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:51.763844 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:51.766455 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:51.767071 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:51.767075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:51.767077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:51.767215 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:51.767489 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:51.767492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:51.767494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:51.767878 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:51.768060 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:51.768358 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 12:15:51.768533 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:15:51.772871 13914 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:51.779780 13914 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:51.805380 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.792475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36521 (local address 127.0.0.1:33600)
0811 12:15:51.795028 (+ 2553us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:51.795043 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:51.795268 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:51.796268 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:51.796277 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:51.796312 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:51.796725 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.796738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.798929 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.798935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.801686 (+ 2751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.801713 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.804176 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.804184 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.804454 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.804511 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 12:15:51.804674 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 12:15:51.804912 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2455,"thread_start_us":2304,"threads_started":1}
I20260811 12:15:51.806453 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.792660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33600 (local address 127.7.39.2:36521)
0811 12:15:51.794621 (+ 1961us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.794632 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.795139 (+ 507us) server_negotiation.cc:407] Connection header received
0811 12:15:51.795490 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.795499 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.795737 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.795924 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:51.796940 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.798722 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.801936 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.803592 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.803703 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.803928 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:51.805856 (+ 1928us) server_negotiation.cc:299] Negotiation successful
0811 12:15:51.806075 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1785,"thread_start_us":1642,"threads_started":1}
I20260811 12:15:51.808326 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.792149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:43251 (local address 127.0.0.1:44982)
0811 12:15:51.793681 (+ 1532us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:51.793723 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:51.793898 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:51.797257 (+ 3359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:51.797262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:51.797291 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:51.797716 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.797727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.801140 (+ 3413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.801146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.804974 (+ 3828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.804991 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.807496 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.807502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.807711 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.807746 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:15:51.807828 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 12:15:51.807968 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1285,"thread_start_us":1127,"threads_started":1}
I20260811 12:15:51.809073 14663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.792661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44982 (local address 127.7.39.3:43251)
0811 12:15:51.795031 (+ 2370us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.795039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.795073 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:15:51.795185 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.795193 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.795490 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.796859 (+ 1369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:51.797950 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.800952 (+ 3002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.805231 (+ 4279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.806888 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.807039 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.807307 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:51.808144 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 12:15:51.808352 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2203,"thread_start_us":2034,"threads_started":1}
I20260811 12:15:51.812124 14336 tablet_service.cc:1467] Processing CreateTablet for tablet cc79ca278fba472592d374de28cfeec6 (DEFAULT_TABLE table=test-table [id=e6bc538c142244cdbdcdde37aa6d1495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:51.813174 14541 tablet_service.cc:1467] Processing CreateTablet for tablet cc79ca278fba472592d374de28cfeec6 (DEFAULT_TABLE table=test-table [id=e6bc538c142244cdbdcdde37aa6d1495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:51.813813 14336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc79ca278fba472592d374de28cfeec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:51.814321 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.795166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45408)
0811 12:15:51.807206 (+ 12040us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:51.807228 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:51.807347 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:51.808486 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:51.808491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:51.808520 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:51.808732 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.808746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.810882 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.810886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.812041 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.812054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.813452 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.813459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.813647 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.813682 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:15:51.813776 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:15:51.814023 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11912,"thread_start_us":12447,"threads_started":1}
I20260811 12:15:51.814330 14665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.795551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45408 (local address 127.7.39.1:40899)
0811 12:15:51.807776 (+ 12225us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.807786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.807819 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:15:51.807937 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.807946 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.808246 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.808931 (+ 685us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:51.809242 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.810721 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.812217 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.813288 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.813409 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.813656 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:51.813837 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 12:15:51.814008 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12014,"thread_start_us":12258,"threads_started":1}
I20260811 12:15:51.814905 14541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc79ca278fba472592d374de28cfeec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:51.818396 14125 tablet_service.cc:1467] Processing CreateTablet for tablet cc79ca278fba472592d374de28cfeec6 (DEFAULT_TABLE table=test-table [id=e6bc538c142244cdbdcdde37aa6d1495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:51.819633 14125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc79ca278fba472592d374de28cfeec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:51.828995 14666 tablet_bootstrap.cc:492] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Bootstrap starting.
I20260811 12:15:51.831517 14667 tablet_bootstrap.cc:492] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Bootstrap starting.
I20260811 12:15:51.833791 14666 tablet_bootstrap.cc:654] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:51.833868 14668 tablet_bootstrap.cc:492] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: Bootstrap starting.
I20260811 12:15:51.835204 14667 tablet_bootstrap.cc:654] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:51.835258 14666 log.cc:824] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:51.836349 14667 log.cc:824] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:51.836885 14668 tablet_bootstrap.cc:654] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:51.838086 14668 log.cc:824] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:51.838630 14666 tablet_bootstrap.cc:492] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: No bootstrap required, opened a new log
I20260811 12:15:51.838935 14666 ts_tablet_manager.cc:1397] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 12:15:51.839537 14667 tablet_bootstrap.cc:492] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: No bootstrap required, opened a new log
I20260811 12:15:51.839816 14667 ts_tablet_manager.cc:1397] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:15:51.840605 14668 tablet_bootstrap.cc:492] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: No bootstrap required, opened a new log
I20260811 12:15:51.840847 14668 ts_tablet_manager.cc:1397] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 12:15:51.848759 14668 raft_consensus.cc:348] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.849189 14668 raft_consensus.cc:374] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:51.849298 14668 raft_consensus.cc:729] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd49096037a04f06aaf07d38b4356b57, State: Initialized, Role: FOLLOWER
I20260811 12:15:51.849485 14666 raft_consensus.cc:348] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.849917 14666 raft_consensus.cc:374] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:51.849911 14668 consensus_queue.cc:260] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.850039 14666 raft_consensus.cc:729] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afc2f76e6dd49bf9dcc1336094b2c24, State: Initialized, Role: FOLLOWER
I20260811 12:15:51.850095 14667 raft_consensus.cc:348] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.850497 14667 raft_consensus.cc:374] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:51.850605 14667 raft_consensus.cc:729] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 596f87909a90403cbb2d659dbc42249e, State: Initialized, Role: FOLLOWER
I20260811 12:15:51.850661 14666 consensus_queue.cc:260] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.851161 14667 consensus_queue.cc:260] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.851963 14429 heartbeater.cc:503] Master 127.7.39.62:46373 was elected leader, sending a full tablet report...
I20260811 12:15:51.852829 14666 ts_tablet_manager.cc:1428] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 12:15:51.857475 14668 ts_tablet_manager.cc:1428] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260811 12:15:51.858459 14636 heartbeater.cc:503] Master 127.7.39.62:46373 was elected leader, sending a full tablet report...
I20260811 12:15:51.859395 14667 ts_tablet_manager.cc:1428] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Time spent starting tablet: real 0.019s user 0.011s sys 0.009s
W20260811 12:15:51.917285 14229 tablet.cc:2369] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:15:51.921363 14684 raft_consensus.cc:482] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:51.921648 14684 raft_consensus.cc:504] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.923308 14684 leader_election.cc:288] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 596f87909a90403cbb2d659dbc42249e (127.7.39.3:43251), 6afc2f76e6dd49bf9dcc1336094b2c24 (127.7.39.2:36521)
I20260811 12:15:51.932408 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.923813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:56843 (local address 127.7.39.2:36521)
0811 12:15:51.924014 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.924020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.924721 (+ 701us) server_negotiation.cc:407] Connection header received
0811 12:15:51.924901 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.924907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.924990 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.925100 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:51.925954 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.927071 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.930063 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.931299 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.931449 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.931552 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:51.932002 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 12:15:51.932142 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:15:51.932408 14663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.923822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:54237 (local address 127.7.39.3:43251)
0811 12:15:51.924014 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.924020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.924780 (+ 760us) server_negotiation.cc:407] Connection header received
0811 12:15:51.924945 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.924949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.925017 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.925099 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:15:51.925954 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.927071 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.930051 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.931332 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.931447 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.931553 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:51.932002 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 12:15:51.932142 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:15:51.932634 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.923741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:43251 (local address 127.7.39.1:54237)
0811 12:15:51.924591 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:51.924612 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:51.924817 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:51.925126 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:51.925130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:51.925208 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:51.925770 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.925788 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.927221 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.927227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.929876 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.929894 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.931562 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.931573 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.931812 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.931860 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 12:15:51.931956 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:15:51.932157 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":222,"spinlock_wait_cycles":67200,"thread_start_us":460,"threads_started":1}
I20260811 12:15:51.932960 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.923642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36521 (local address 127.7.39.1:56843)
0811 12:15:51.924597 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:51.924612 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:51.924769 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:51.925126 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:51.925130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:51.925208 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:15:51.925810 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.925824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.927222 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.927227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.929876 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:51.929894 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.931597 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.931603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:51.931806 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.931872 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 12:15:51.931953 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:15:51.932157 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":221,"spinlock_wait_cycles":19712,"thread_start_us":599,"threads_started":1}
I20260811 12:15:51.933918 14363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc79ca278fba472592d374de28cfeec6" candidate_uuid: "dd49096037a04f06aaf07d38b4356b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" is_pre_election: true
I20260811 12:15:51.933897 14566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc79ca278fba472592d374de28cfeec6" candidate_uuid: "dd49096037a04f06aaf07d38b4356b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "596f87909a90403cbb2d659dbc42249e" is_pre_election: true
I20260811 12:15:51.934501 14566 raft_consensus.cc:2454] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd49096037a04f06aaf07d38b4356b57 in term 0.
I20260811 12:15:51.934501 14363 raft_consensus.cc:2454] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd49096037a04f06aaf07d38b4356b57 in term 0.
I20260811 12:15:51.935238 14043 leader_election.cc:302] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 596f87909a90403cbb2d659dbc42249e, 6afc2f76e6dd49bf9dcc1336094b2c24, dd49096037a04f06aaf07d38b4356b57; no voters:
I20260811 12:15:51.935703 14684 raft_consensus.cc:2781] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:51.935855 14684 raft_consensus.cc:482] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:51.935951 14684 raft_consensus.cc:3037] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:51.938186 14684 raft_consensus.cc:504] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.938773 14684 leader_election.cc:288] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [CANDIDATE]: Term 1 election: Requested vote from peers 596f87909a90403cbb2d659dbc42249e (127.7.39.3:43251), 6afc2f76e6dd49bf9dcc1336094b2c24 (127.7.39.2:36521)
I20260811 12:15:51.939297 14363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc79ca278fba472592d374de28cfeec6" candidate_uuid: "dd49096037a04f06aaf07d38b4356b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24"
I20260811 12:15:51.939493 14363 raft_consensus.cc:3037] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:51.939700 14566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc79ca278fba472592d374de28cfeec6" candidate_uuid: "dd49096037a04f06aaf07d38b4356b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "596f87909a90403cbb2d659dbc42249e"
I20260811 12:15:51.939921 14566 raft_consensus.cc:3037] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:51.942955 14363 raft_consensus.cc:2454] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd49096037a04f06aaf07d38b4356b57 in term 1.
I20260811 12:15:51.943084 14566 raft_consensus.cc:2454] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd49096037a04f06aaf07d38b4356b57 in term 1.
I20260811 12:15:51.943545 14043 leader_election.cc:302] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 596f87909a90403cbb2d659dbc42249e, dd49096037a04f06aaf07d38b4356b57; no voters:
I20260811 12:15:51.943887 14684 raft_consensus.cc:2781] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:51.944372 14684 raft_consensus.cc:686] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 1 LEADER]: Becoming Leader. State: Replica: dd49096037a04f06aaf07d38b4356b57, State: Running, Role: LEADER
I20260811 12:15:51.944924 14684 consensus_queue.cc:237] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } }
I20260811 12:15:51.949592 13910 catalog_manager.cc:5495] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd49096037a04f06aaf07d38b4356b57 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "dd49096037a04f06aaf07d38b4356b57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd49096037a04f06aaf07d38b4356b57" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40899 } health_report { overall_health: HEALTHY } } }
W20260811 12:15:51.951517 14432 tablet.cc:2369] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:15:51.973001 14637 tablet.cc:2369] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:15:52.000716 14665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:51.995271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.7.39.1:40899)
0811 12:15:51.995465 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:51.995471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:51.995656 (+ 185us) server_negotiation.cc:407] Connection header received
0811 12:15:51.995879 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:51.995884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:51.995965 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:51.996099 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:15:51.996484 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.997234 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.998578 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:51.998990 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:51.999102 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:51.999599 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:15:52.000137 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:15:52.000274 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:52.000361 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 12:15:52.000447 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:15:52.010350 14566 raft_consensus.cc:1261] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 1 FOLLOWER]: Refusing update from remote peer dd49096037a04f06aaf07d38b4356b57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:52.010370 14363 raft_consensus.cc:1261] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 1 FOLLOWER]: Refusing update from remote peer dd49096037a04f06aaf07d38b4356b57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:52.011446 14684 consensus_queue.cc:1035] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "596f87909a90403cbb2d659dbc42249e" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:52.011818 14691 consensus_queue.cc:1035] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6afc2f76e6dd49bf9dcc1336094b2c24" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:52.053900 13910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:52.054215 13910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:52.058643 13910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38d7071dfbad4bd8ad3ca8d6640a7360: Sending DeleteTablet for 3 replicas of tablet cc79ca278fba472592d374de28cfeec6
I20260811 12:15:52.059947 14336 tablet_service.cc:1514] Processing DeleteTablet for tablet cc79ca278fba472592d374de28cfeec6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:52 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:33600
I20260811 12:15:52.059988 14541 tablet_service.cc:1514] Processing DeleteTablet for tablet cc79ca278fba472592d374de28cfeec6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:52 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:44982
I20260811 12:15:52.066203 14125 tablet_service.cc:1514] Processing DeleteTablet for tablet cc79ca278fba472592d374de28cfeec6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:52 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:45408
I20260811 12:15:52.070492 14710 tablet_replica.cc:330] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: stopping tablet replica
I20260811 12:15:52.071201 14715 tablet_replica.cc:330] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: stopping tablet replica
I20260811 12:15:52.072453 14715 raft_consensus.cc:2229] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:15:52.073012 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 14015
I20260811 12:15:52.073071 14710 raft_consensus.cc:2229] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:52.074307 14710 raft_consensus.cc:2258] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:52.075798 14715 raft_consensus.cc:2258] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:52.079937 14715 ts_tablet_manager.cc:1905] T cc79ca278fba472592d374de28cfeec6 P dd49096037a04f06aaf07d38b4356b57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:52.081203 14712 tablet_replica.cc:330] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: stopping tablet replica
I20260811 12:15:52.081840 14712 raft_consensus.cc:2229] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:52.082525 14712 raft_consensus.cc:2258] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:52.083899 14710 ts_tablet_manager.cc:1905] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:52.086424 14712 ts_tablet_manager.cc:1905] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:52.093014 14710 ts_tablet_manager.cc:1918] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:15:52.093251 14710 log.cc:1196] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/cc79ca278fba472592d374de28cfeec6
I20260811 12:15:52.093839 14710 ts_tablet_manager.cc:1939] T cc79ca278fba472592d374de28cfeec6 P 6afc2f76e6dd49bf9dcc1336094b2c24: Deleting consensus metadata
Aug 11 12:15:52 dist-test-slave-rmhg krb5kdc[13848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:15:52.096174 13889 catalog_manager.cc:4841] TS 6afc2f76e6dd49bf9dcc1336094b2c24 (127.7.39.2:36521): tablet cc79ca278fba472592d374de28cfeec6 (table test-table [id=e6bc538c142244cdbdcdde37aa6d1495]) successfully deleted
I20260811 12:15:52.102388 14712 ts_tablet_manager.cc:1918] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:15:52.102613 14712 log.cc:1196] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/cc79ca278fba472592d374de28cfeec6
I20260811 12:15:52.103220 14712 ts_tablet_manager.cc:1939] T cc79ca278fba472592d374de28cfeec6 P 596f87909a90403cbb2d659dbc42249e: Deleting consensus metadata
I20260811 12:15:52.103945 14665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.075168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.7.39.1:40899)
0811 12:15:52.075346 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:52.075352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:52.075376 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:15:52.075439 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:52.075444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:52.075527 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:52.075646 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:52.079382 (+ 3736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:52.080538 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:52.085279 (+ 4741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:52.086002 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:52.086114 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:52.096009 (+ 9895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:52.096029 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:52.096047 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:52.096119 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:52.100168 (+ 4049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:52.101130 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:52.101139 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:52.101143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:52.101346 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:52.101821 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:52.101826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:52.101830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:52.102344 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:52.102651 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:52.103314 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 12:15:52.103567 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:15:52.105293 13890 catalog_manager.cc:4841] TS 596f87909a90403cbb2d659dbc42249e (127.7.39.3:43251): tablet cc79ca278fba472592d374de28cfeec6 (table test-table [id=e6bc538c142244cdbdcdde37aa6d1495]) successfully deleted
I20260811 12:15:52.105597 14216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:52.208277 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 14015
W20260811 12:15:52.241226 13889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.39.1:40899 (error 108)
I20260811 12:15:52.241497 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 14236
I20260811 12:15:52.243498 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.242974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45440)
0811 12:15:52.243170 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:52.243337 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 12:15:52.243784 13889 catalog_manager.cc:4584] TS dd49096037a04f06aaf07d38b4356b57 (127.7.39.1:40899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc79ca278fba472592d374de28cfeec6: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Aug 11 12:15:52 dist-test-slave-rmhg krb5kdc[13848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:15:52.256978 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.242417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33602 (local address 127.7.39.2:36521)
0811 12:15:52.242628 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:52.242635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:52.242730 (+ 95us) server_negotiation.cc:407] Connection header received
0811 12:15:52.242934 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:52.242939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:52.243021 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:52.243119 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:52.244674 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:52.245571 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:52.246823 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:52.247242 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:52.247354 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:52.251359 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:52.251376 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:52.251395 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:52.251436 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:52.254221 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:52.254907 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:52.254915 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:52.254918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:52.255060 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:52.255439 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:52.255443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:52.255446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:52.255845 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:52.256130 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:52.256425 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:15:52.256652 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:15:52.257786 14423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:52.297595 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.297108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45442)
0811 12:15:52.297345 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:52.297441 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 12:15:52.298764 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.298396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45452)
0811 12:15:52.298524 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:52.298601 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 12:15:52.348707 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 14236
I20260811 12:15:52.359258 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.358799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45458)
0811 12:15:52.359003 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:52.359097 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 12:15:52.360543 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.360197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45460)
0811 12:15:52.360344 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:52.360420 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:15:52.373759 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 14443
Aug 11 12:15:52 dist-test-slave-rmhg krb5kdc[13848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:15:52.390836 14663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.374548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44990 (local address 127.7.39.3:43251)
0811 12:15:52.374714 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:52.374718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:52.374740 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:15:52.374888 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:52.374892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:52.374960 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:52.375048 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:52.376675 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:52.377874 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:52.379558 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:52.380207 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:52.380346 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:52.385471 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:52.385490 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:52.385508 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:15:52.385551 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:52.388389 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:52.389040 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:52.389047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:52.389050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:52.389257 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:15:52.389577 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:15:52.389580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:15:52.389582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:15:52.389972 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:52.390227 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:52.390405 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 12:15:52.390577 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:15:52.391546 14626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:52.441810 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.441330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45466)
0811 12:15:52.441555 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:52.441651 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 12:15:52.442895 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.442451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45482)
0811 12:15:52.442673 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:52.442749 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 12:15:52.480942 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 14443
I20260811 12:15:52.507493 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 13868
I20260811 12:15:52.508674 13967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:52.618700 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:52.618112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40899 (local address 127.0.0.1:45488)
0811 12:15:52.618347 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:52.618459 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 12:15:52.618949 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 13868
2026-08-11T12:15:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4548 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T12:15:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:52Z Disabled control of system clock
I20260811 12:15:52.699465 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45867
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:45867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 12:15:53.033087 14734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:53.033416 14734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:53.033510 14734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:53.040632 14734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:53.040758 14734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:53.040800 14734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:53.040869 14734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:53.040933 14734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:15:53.050221 14734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:45867
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:53.051604 14734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:53.053432 14734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:53.064841 14746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:53.064839 14749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:53.064987 14744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:53.065742 14734 server_base.cc:1034] running on GCE node
I20260811 12:15:53.066609 14734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:53.068117 14734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:53.069319 14734 hybrid_clock.cc:648] HybridClock initialized: now 1786450553069264 us; error 58 us; skew 500 ppm
I20260811 12:15:53.069669 14734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:15:53.071966 14734 webserver.cc:460] Webserver started at http://127.7.39.62:44673/ using document root <none> and password file <none>
I20260811 12:15:53.072665 14734 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:53.072773 14734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:53.073103 14734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:53.076126 14734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "ed01b34f12184a05bcba5c784c35ec6f"
format_stamp: "Formatted at 2026-08-11 12:15:53 on dist-test-slave-rmhg"
server_key: "ce9f5fbf805d5ccf4d7c67f6d4827c3a"
server_key_iv: "c2222be3e1ff29f03867cc16d109215e"
server_key_version: "encryptionkey@0"
I20260811 12:15:53.076912 14734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "ed01b34f12184a05bcba5c784c35ec6f"
format_stamp: "Formatted at 2026-08-11 12:15:53 on dist-test-slave-rmhg"
server_key: "ce9f5fbf805d5ccf4d7c67f6d4827c3a"
server_key_iv: "c2222be3e1ff29f03867cc16d109215e"
server_key_version: "encryptionkey@0"
I20260811 12:15:53.082165 14734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:15:53.085652 14756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:53.087113 14734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 12:15:53.087312 14734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "ed01b34f12184a05bcba5c784c35ec6f"
format_stamp: "Formatted at 2026-08-11 12:15:53 on dist-test-slave-rmhg"
server_key: "ce9f5fbf805d5ccf4d7c67f6d4827c3a"
server_key_iv: "c2222be3e1ff29f03867cc16d109215e"
server_key_version: "encryptionkey@0"
I20260811 12:15:53.087589 14734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:53.093679 14734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:53.094431 14734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:53.094688 14734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:53.112365 14734 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:45867
I20260811 12:15:53.112385 14833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:45867 every 8 connection(s)
I20260811 12:15:53.113826 14734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:53.117836 14836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:53.122740 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 14734
I20260811 12:15:53.122920 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:53.123562 7324 external_mini_cluster.cc:1428] Setting key e4b57595aa7776e567564ddcfea85610
I20260811 12:15:53.130125 14836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f: Bootstrap starting.
I20260811 12:15:53.133652 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:53.125791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.7.39.62:45867)
0811 12:15:53.127009 (+ 1218us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:53.127024 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:53.127080 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:15:53.129397 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:53.129429 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:53.130008 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:53.130489 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:53.131519 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:53.131528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:53.131546 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:53.131564 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:53.131643 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:53.131665 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:53.131905 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:53.132059 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 12:15:53.132392 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":369,"threads_started":1}
I20260811 12:15:53.134256 14836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:53.135349 14836 log.cc:824] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:53.138350 14836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f: No bootstrap required, opened a new log
I20260811 12:15:53.145781 14836 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed01b34f12184a05bcba5c784c35ec6f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45867 } }
I20260811 12:15:53.146167 14836 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:53.146262 14836 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed01b34f12184a05bcba5c784c35ec6f, State: Initialized, Role: FOLLOWER
I20260811 12:15:53.146865 14836 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed01b34f12184a05bcba5c784c35ec6f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45867 } }
I20260811 12:15:53.147097 14836 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:53.147224 14836 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:53.147385 14836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:53.149863 14836 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed01b34f12184a05bcba5c784c35ec6f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45867 } }
I20260811 12:15:53.150382 14836 leader_election.cc:302] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed01b34f12184a05bcba5c784c35ec6f; no voters:
I20260811 12:15:53.151001 14836 leader_election.cc:288] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:53.151129 14842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:53.151791 14842 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [term 1 LEADER]: Becoming Leader. State: Replica: ed01b34f12184a05bcba5c784c35ec6f, State: Running, Role: LEADER
I20260811 12:15:53.152328 14842 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed01b34f12184a05bcba5c784c35ec6f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45867 } }
I20260811 12:15:53.153097 14836 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:53.155392 14843 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed01b34f12184a05bcba5c784c35ec6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed01b34f12184a05bcba5c784c35ec6f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45867 } } }
I20260811 12:15:53.155809 14843 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:53.155953 14845 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed01b34f12184a05bcba5c784c35ec6f. Latest consensus state: current_term: 1 leader_uuid: "ed01b34f12184a05bcba5c784c35ec6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed01b34f12184a05bcba5c784c35ec6f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45867 } } }
I20260811 12:15:53.156314 14845 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:53.163251 14856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:53.168754 14856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:53.177796 14856 catalog_manager.cc:1347] Generated new cluster ID: b7cdb9171a8e4e4cafdd2ba1f02fce05
I20260811 12:15:53.177929 14856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:53.190400 14856 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:53.190864 14856 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:53.198820 14856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f: Generated new TSK 0
I20260811 12:15:53.199707 14856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:15:53.209563 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45867
--builtin_ntp_servers=127.7.39.20:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 12:15:53.555161 14872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:53.555491 14872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:53.555634 14872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:53.563267 14872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:53.563403 14872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:53.563545 14872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:15:53.573028 14872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:53.574409 14872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:53.576297 14872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:53.589668 14878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:53.589737 14882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:53.590273 14877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:53.590668 14872 server_base.cc:1034] running on GCE node
I20260811 12:15:53.591323 14872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:53.592373 14872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:53.593605 14872 hybrid_clock.cc:648] HybridClock initialized: now 1786450553593522 us; error 63 us; skew 500 ppm
I20260811 12:15:53.593926 14872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:15:53.599192 14872 webserver.cc:460] Webserver started at http://127.7.39.1:45447/ using document root <none> and password file <none>
I20260811 12:15:53.599915 14872 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:53.600008 14872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:53.600289 14872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:53.603415 14872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "21f3a652f2bc402fbdc540ac8002410e"
format_stamp: "Formatted at 2026-08-11 12:15:53 on dist-test-slave-rmhg"
server_key: "f00da802b5c6a5d4e1f7ba350c04887b"
server_key_iv: "cbc1cef28a770d75bd42ca9718c6b035"
server_key_version: "encryptionkey@0"
I20260811 12:15:53.604199 14872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "21f3a652f2bc402fbdc540ac8002410e"
format_stamp: "Formatted at 2026-08-11 12:15:53 on dist-test-slave-rmhg"
server_key: "f00da802b5c6a5d4e1f7ba350c04887b"
server_key_iv: "cbc1cef28a770d75bd42ca9718c6b035"
server_key_version: "encryptionkey@0"
I20260811 12:15:53.609536 14872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 12:15:53.613176 14889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:53.614549 14872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:53.614703 14872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "21f3a652f2bc402fbdc540ac8002410e"
format_stamp: "Formatted at 2026-08-11 12:15:53 on dist-test-slave-rmhg"
server_key: "f00da802b5c6a5d4e1f7ba350c04887b"
server_key_iv: "cbc1cef28a770d75bd42ca9718c6b035"
server_key_version: "encryptionkey@0"
I20260811 12:15:53.614904 14872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:53.620519 14872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:53.621377 14872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:53.621620 14872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:53.622483 14872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:53.623898 14872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:53.623986 14872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:53.624069 14872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:53.624131 14872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:53.655838 14872 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:42925
I20260811 12:15:53.655893 15067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:42925 every 8 connection(s)
I20260811 12:15:53.657241 14872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:15:53.664713 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 14872
I20260811 12:15:53.664897 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:15:53.665380 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:53.659763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:43371 (local address 127.7.39.62:45867)
0811 12:15:53.660205 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:53.660209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:53.661408 (+ 1199us) server_negotiation.cc:407] Connection header received
0811 12:15:53.663119 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:53.663125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:53.663191 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:53.663288 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:53.664600 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:53.664607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:53.664611 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:53.664630 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:53.664673 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:53.664685 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:53.664797 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:53.665004 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 12:15:53.665073 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":219,"threads_started":1}
I20260811 12:15:53.665604 7324 external_mini_cluster.cc:1428] Setting key da2782289fec8ffecbdd901f262ea251
I20260811 12:15:53.666328 15072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:53.660283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45867 (local address 127.7.39.1:43371)
0811 12:15:53.661253 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:53.661299 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:53.662842 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:53.663534 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:53.663542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:53.663641 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:53.664285 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:53.664288 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:53.664325 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:53.664411 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:53.664776 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:53.664786 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:53.664794 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:15:53.665072 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 12:15:53.665343 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":281,"threads_started":1}
I20260811 12:15:53.667894 15069 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45867
I20260811 12:15:53.668179 15069 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:53.668514 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45867
--builtin_ntp_servers=127.7.39.20:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 12:15:53.668794 15069 heartbeater.cc:511] Master 127.7.39.62:45867 requested a full tablet report, sending...
I20260811 12:15:53.671025 14776 ts_manager.cc:194] Registered new tserver with Master: 21f3a652f2bc402fbdc540ac8002410e (127.7.39.1:42925)
W20260811 12:15:54.002183 15073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:54.002499 15073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:54.002630 15073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:54.009908 15073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:54.010053 15073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:54.010222 15073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:15:54.019531 15073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:54.020848 15073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:54.022823 15073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:54.035910 15078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:54.035826 15079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:54.036142 15082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:54.036804 15073 server_base.cc:1034] running on GCE node
I20260811 12:15:54.037521 15073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:54.038515 15073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:54.039726 15073 hybrid_clock.cc:648] HybridClock initialized: now 1786450554039675 us; error 45 us; skew 500 ppm
I20260811 12:15:54.040052 15073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:15:54.042368 15073 webserver.cc:460] Webserver started at http://127.7.39.2:33417/ using document root <none> and password file <none>
I20260811 12:15:54.043079 15073 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:54.043198 15073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:54.043510 15073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:54.046620 15073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "d86dfadd00ba46bd89a26346dcb48124"
format_stamp: "Formatted at 2026-08-11 12:15:54 on dist-test-slave-rmhg"
server_key: "bdd12f6c5b01f6e094b0ec88cdbed43b"
server_key_iv: "4f8697fc81d040fa2055c15267fd556a"
server_key_version: "encryptionkey@0"
I20260811 12:15:54.047526 15073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "d86dfadd00ba46bd89a26346dcb48124"
format_stamp: "Formatted at 2026-08-11 12:15:54 on dist-test-slave-rmhg"
server_key: "bdd12f6c5b01f6e094b0ec88cdbed43b"
server_key_iv: "4f8697fc81d040fa2055c15267fd556a"
server_key_version: "encryptionkey@0"
I20260811 12:15:54.052997 15073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:15:54.056777 15090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:54.058331 15073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:54.058495 15073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "d86dfadd00ba46bd89a26346dcb48124"
format_stamp: "Formatted at 2026-08-11 12:15:54 on dist-test-slave-rmhg"
server_key: "bdd12f6c5b01f6e094b0ec88cdbed43b"
server_key_iv: "4f8697fc81d040fa2055c15267fd556a"
server_key_version: "encryptionkey@0"
I20260811 12:15:54.058732 15073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:54.064450 15073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:54.065326 15073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:54.065583 15073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:54.066540 15073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:54.067956 15073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:54.068056 15073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:54.068172 15073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:54.068240 15073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:54.099316 15073 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:36229
I20260811 12:15:54.099377 15260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:36229 every 8 connection(s)
I20260811 12:15:54.100713 15073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:15:54.101691 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 15073
I20260811 12:15:54.101845 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:15:54.102453 7324 external_mini_cluster.cc:1428] Setting key 97fb0546712bdccabe9ac6a2e794fe11
I20260811 12:15:54.106033 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45867
--builtin_ntp_servers=127.7.39.20:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 12:15:54.111099 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.104354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:47215 (local address 127.7.39.62:45867)
0811 12:15:54.104538 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.104543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.106518 (+ 1975us) server_negotiation.cc:407] Connection header received
0811 12:15:54.108800 (+ 2282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.108805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.108861 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.108951 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.110459 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.110464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.110467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.110483 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.110518 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.110529 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.110640 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.110815 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.110867 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:15:54.112385 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.105178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45867 (local address 127.7.39.2:47215)
0811 12:15:54.106341 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.106397 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.108490 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.109254 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.109265 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.109393 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.110103 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.110106 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.110157 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.110274 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.110619 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.110628 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.110635 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.110905 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.111179 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":319,"threads_started":1}
I20260811 12:15:54.114087 15261 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45867
I20260811 12:15:54.114405 15261 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:54.115088 15261 heartbeater.cc:511] Master 127.7.39.62:45867 requested a full tablet report, sending...
I20260811 12:15:54.116493 14776 ts_manager.cc:194] Registered new tserver with Master: d86dfadd00ba46bd89a26346dcb48124 (127.7.39.2:36229)
W20260811 12:15:54.449676 15267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:54.450018 15267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:54.450148 15267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:54.457870 15267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:54.458007 15267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:54.458200 15267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:15:54.467900 15267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:54.469413 15267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:54.471302 15267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:54.485926 15273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:54.485927 15274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:54.486308 15276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:54.487299 15267 server_base.cc:1034] running on GCE node
I20260811 12:15:54.487987 15267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:54.489009 15267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:54.490267 15267 hybrid_clock.cc:648] HybridClock initialized: now 1786450554490201 us; error 67 us; skew 500 ppm
I20260811 12:15:54.490629 15267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:15:54.493135 15267 webserver.cc:460] Webserver started at http://127.7.39.3:33771/ using document root <none> and password file <none>
I20260811 12:15:54.493968 15267 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:54.494096 15267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:54.494434 15267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:54.497663 15267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "7f10c649ff4b4a8a98a36434ebad6067"
format_stamp: "Formatted at 2026-08-11 12:15:54 on dist-test-slave-rmhg"
server_key: "739dd8f41c9c1a4e685309526f5c586a"
server_key_iv: "0458789b747347a31bb762dfaa33673f"
server_key_version: "encryptionkey@0"
I20260811 12:15:54.498546 15267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "7f10c649ff4b4a8a98a36434ebad6067"
format_stamp: "Formatted at 2026-08-11 12:15:54 on dist-test-slave-rmhg"
server_key: "739dd8f41c9c1a4e685309526f5c586a"
server_key_iv: "0458789b747347a31bb762dfaa33673f"
server_key_version: "encryptionkey@0"
I20260811 12:15:54.504483 15267 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 12:15:54.508455 15289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:54.510064 15267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:15:54.510291 15267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "7f10c649ff4b4a8a98a36434ebad6067"
format_stamp: "Formatted at 2026-08-11 12:15:54 on dist-test-slave-rmhg"
server_key: "739dd8f41c9c1a4e685309526f5c586a"
server_key_iv: "0458789b747347a31bb762dfaa33673f"
server_key_version: "encryptionkey@0"
I20260811 12:15:54.510542 15267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:54.516932 15267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:54.517886 15267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:54.518172 15267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:54.519263 15267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:15:54.520757 15267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:15:54.520857 15267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:54.520974 15267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:15:54.521051 15267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:54.553066 15267 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:43535
I20260811 12:15:54.553176 15457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:43535 every 8 connection(s)
I20260811 12:15:54.554533 15267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:15:54.563251 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.557270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:57689 (local address 127.7.39.62:45867)
0811 12:15:54.557460 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.557465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.559065 (+ 1600us) server_negotiation.cc:407] Connection header received
0811 12:15:54.560897 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.560903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.560976 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.561071 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.562501 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.562506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.562509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.562529 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.562565 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.562577 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.562697 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.562884 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.562961 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:15:54.563742 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 15267
I20260811 12:15:54.563898 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:15:54.564337 7324 external_mini_cluster.cc:1428] Setting key 59b7f2de36b630644279237845767240
I20260811 12:15:54.564354 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.557865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45867 (local address 127.7.39.3:57689)
0811 12:15:54.558900 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.558939 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.560600 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.561364 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.561374 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.561477 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.562149 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.562151 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.562204 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.562304 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.562667 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.562679 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.562687 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.562963 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.563260 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":350,"threads_started":1}
I20260811 12:15:54.565974 15459 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45867
I20260811 12:15:54.566326 15459 heartbeater.cc:464] Registering TS with master...
I20260811 12:15:54.567006 15459 heartbeater.cc:511] Master 127.7.39.62:45867 requested a full tablet report, sending...
I20260811 12:15:54.568436 14776 ts_manager.cc:194] Registered new tserver with Master: 7f10c649ff4b4a8a98a36434ebad6067 (127.7.39.3:43535)
I20260811 12:15:54.580232 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:15:54.587445 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.584959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60830 (local address 127.7.39.62:45867)
0811 12:15:54.585174 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.585181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.585718 (+ 537us) server_negotiation.cc:407] Connection header received
0811 12:15:54.585865 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.585869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.585929 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.586024 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.586971 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.586975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.586978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.586991 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.587020 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.587030 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.587138 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.587173 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.587223 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:15:54.591817 14776 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:15:54.598933 14776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:15:54.629855 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.614332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.7.39.2:36229)
0811 12:15:54.614954 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.614976 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.624854 (+ 9878us) server_negotiation.cc:407] Connection header received
0811 12:15:54.625010 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.625019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.625341 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.625522 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.628713 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.628725 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.628744 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.628777 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.628846 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.628868 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.628986 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.629368 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.629485 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":863,"threads_started":1}
I20260811 12:15:54.630740 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.612176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36229 (local address 127.0.0.1:59848)
0811 12:15:54.624638 (+ 12462us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.624689 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.624880 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.627302 (+ 2422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.627312 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.627431 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.628343 (+ 912us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.628346 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.628378 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.628477 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.629091 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.629104 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.629113 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.630157 (+ 1044us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.630267 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12082,"thread_start_us":14198,"threads_started":1}
I20260811 12:15:54.631309 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.614094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:43535 (local address 127.0.0.1:50704)
0811 12:15:54.626680 (+ 12586us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.626699 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.626799 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.629247 (+ 2448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.629253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.629282 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.629963 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.629967 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.630175 (+ 208us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.630247 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.630650 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.630655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.630661 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.630828 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.630882 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12458,"thread_start_us":9986,"threads_started":1}
I20260811 12:15:54.632025 15485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.614660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50704 (local address 127.7.39.3:43535)
0811 12:15:54.627376 (+ 12716us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.627386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.627427 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:15:54.627506 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.627514 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.627743 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.627918 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.630430 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.630436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.630447 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.630462 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.630518 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.630534 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.630687 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.631547 (+ 860us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.631668 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12534,"thread_start_us":14304,"threads_started":1}
I20260811 12:15:54.636107 15369 tablet_service.cc:1467] Processing CreateTablet for tablet 592de7b77985441b9167bd442e5d1f40 (DEFAULT_TABLE table=test-table [id=efb4d73fec584b168348f7224ad7a7eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:54.636757 15172 tablet_service.cc:1467] Processing CreateTablet for tablet 592de7b77985441b9167bd442e5d1f40 (DEFAULT_TABLE table=test-table [id=efb4d73fec584b168348f7224ad7a7eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:54.637791 15369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592de7b77985441b9167bd442e5d1f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:54.638237 15172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592de7b77985441b9167bd442e5d1f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:54.648588 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.628780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.0.0.1:42380)
0811 12:15:54.630994 (+ 2214us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.631016 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.631131 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.632268 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.632273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.632297 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.647001 (+ 14704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.647006 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.647030 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.647108 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.647783 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.647790 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.647795 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.647955 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.648034 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1966,"thread_start_us":1094,"threads_started":1}
I20260811 12:15:54.649588 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.627161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.7.39.1:42925)
0811 12:15:54.627760 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.627769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.631275 (+ 3506us) server_negotiation.cc:407] Connection header received
0811 12:15:54.631358 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.631368 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.631723 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.631873 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.647375 (+ 15502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.647388 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.647407 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.647444 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.647516 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.647541 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.647706 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.649055 (+ 1349us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.649216 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":1254,"threads_started":1}
I20260811 12:15:54.654270 14960 tablet_service.cc:1467] Processing CreateTablet for tablet 592de7b77985441b9167bd442e5d1f40 (DEFAULT_TABLE table=test-table [id=efb4d73fec584b168348f7224ad7a7eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:15:54.655717 14960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592de7b77985441b9167bd442e5d1f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:54.658767 15488 tablet_bootstrap.cc:492] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Bootstrap starting.
I20260811 12:15:54.662909 15488 tablet_bootstrap.cc:654] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:54.664369 15488 log.cc:824] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:54.667624 15489 tablet_bootstrap.cc:492] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Bootstrap starting.
I20260811 12:15:54.669510 15488 tablet_bootstrap.cc:492] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: No bootstrap required, opened a new log
I20260811 12:15:54.669818 15488 ts_tablet_manager.cc:1397] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 12:15:54.672276 15489 tablet_bootstrap.cc:654] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:54.673877 15489 log.cc:824] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:54.674157 15069 heartbeater.cc:503] Master 127.7.39.62:45867 was elected leader, sending a full tablet report...
I20260811 12:15:54.677811 15492 tablet_bootstrap.cc:492] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: Bootstrap starting.
I20260811 12:15:54.680004 15488 raft_consensus.cc:348] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.680519 15488 raft_consensus.cc:374] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:54.680624 15488 raft_consensus.cc:729] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86dfadd00ba46bd89a26346dcb48124, State: Initialized, Role: FOLLOWER
I20260811 12:15:54.681452 15488 consensus_queue.cc:260] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.682732 15492 tablet_bootstrap.cc:654] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:54.684260 15492 log.cc:824] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:54.688761 15261 heartbeater.cc:503] Master 127.7.39.62:45867 was elected leader, sending a full tablet report...
I20260811 12:15:54.689873 15488 ts_tablet_manager.cc:1428] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
I20260811 12:15:54.694591 15489 tablet_bootstrap.cc:492] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: No bootstrap required, opened a new log
I20260811 12:15:54.695407 15489 ts_tablet_manager.cc:1397] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.005s
I20260811 12:15:54.695748 15492 tablet_bootstrap.cc:492] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: No bootstrap required, opened a new log
I20260811 12:15:54.696041 15492 ts_tablet_manager.cc:1397] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.000s
I20260811 12:15:54.706408 15492 raft_consensus.cc:348] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.706964 15492 raft_consensus.cc:374] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:54.706542 15489 raft_consensus.cc:348] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.707068 15492 raft_consensus.cc:729] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21f3a652f2bc402fbdc540ac8002410e, State: Initialized, Role: FOLLOWER
I20260811 12:15:54.707301 15489 raft_consensus.cc:374] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:54.707465 15489 raft_consensus.cc:729] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f10c649ff4b4a8a98a36434ebad6067, State: Initialized, Role: FOLLOWER
I20260811 12:15:54.707785 15492 consensus_queue.cc:260] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.708254 15489 consensus_queue.cc:260] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.709632 15492 ts_tablet_manager.cc:1428] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260811 12:15:54.709704 15459 heartbeater.cc:503] Master 127.7.39.62:45867 was elected leader, sending a full tablet report...
I20260811 12:15:54.710239 15489 ts_tablet_manager.cc:1428] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260811 12:15:54.806244 15460 tablet.cc:2369] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:15:54.853621 15263 tablet.cc:2369] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:15:54.909848 15070 tablet.cc:2369] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:15:54.951211 15495 raft_consensus.cc:482] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:54.951560 15495 raft_consensus.cc:504] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.953866 15495 leader_election.cc:288] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f10c649ff4b4a8a98a36434ebad6067 (127.7.39.3:43535), 21f3a652f2bc402fbdc540ac8002410e (127.7.39.1:42925)
I20260811 12:15:54.955217 15498 raft_consensus.cc:482] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:15:54.955570 15498 raft_consensus.cc:504] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.957479 15485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.954516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:42385 (local address 127.7.39.3:43535)
0811 12:15:54.954727 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.954733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.955479 (+ 746us) server_negotiation.cc:407] Connection header received
0811 12:15:54.955709 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.955720 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.955796 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.955908 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.956867 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.956874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.956879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.956898 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.956939 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.956951 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.957092 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.957138 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.957228 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:15:54.957685 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.954291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:43535 (local address 127.7.39.2:42385)
0811 12:15:54.955353 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.955373 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.955571 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.955929 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.955934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.955958 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.956653 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.956657 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.956675 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.956731 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.957041 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.957045 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.957049 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.957214 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.957271 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"thread_start_us":725,"threads_started":1}
I20260811 12:15:54.957828 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.954488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:32821 (local address 127.7.39.1:42925)
0811 12:15:54.954701 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.954708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.955468 (+ 760us) server_negotiation.cc:407] Connection header received
0811 12:15:54.955812 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.955818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.955886 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.955961 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.957045 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.957051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.957054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.957082 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.957127 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.957142 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.957284 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.957505 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.957571 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:15:54.958930 15498 leader_election.cc:288] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86dfadd00ba46bd89a26346dcb48124 (127.7.39.2:36229), 21f3a652f2bc402fbdc540ac8002410e (127.7.39.1:42925)
I20260811 12:15:54.959112 15398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592de7b77985441b9167bd442e5d1f40" candidate_uuid: "d86dfadd00ba46bd89a26346dcb48124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f10c649ff4b4a8a98a36434ebad6067" is_pre_election: true
I20260811 12:15:54.959476 15500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.954263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.7.39.2:32821)
0811 12:15:54.955343 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.955365 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.955630 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.956187 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.956192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.956221 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.956832 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.956835 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.956852 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.956897 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.957339 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.957345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.957350 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.958078 (+ 728us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.958142 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"spinlock_wait_cycles":27008,"thread_start_us":1383,"threads_started":1}
I20260811 12:15:54.959671 15398 raft_consensus.cc:2454] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86dfadd00ba46bd89a26346dcb48124 in term 0.
I20260811 12:15:54.960464 15094 leader_election.cc:302] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f10c649ff4b4a8a98a36434ebad6067, d86dfadd00ba46bd89a26346dcb48124; no voters:
I20260811 12:15:54.960592 14999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592de7b77985441b9167bd442e5d1f40" candidate_uuid: "d86dfadd00ba46bd89a26346dcb48124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f3a652f2bc402fbdc540ac8002410e" is_pre_election: true
I20260811 12:15:54.960958 15495 raft_consensus.cc:2781] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:54.961139 15495 raft_consensus.cc:482] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:54.961293 15495 raft_consensus.cc:3037] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:54.961268 14999 raft_consensus.cc:2454] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86dfadd00ba46bd89a26346dcb48124 in term 0.
I20260811 12:15:54.961683 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.958060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36229 (local address 127.7.39.3:37765)
0811 12:15:54.958360 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.958381 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.958461 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.959162 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.959167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.959196 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.960430 (+ 1234us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.960438 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.960460 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.960515 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.961258 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.961263 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.961268 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.961380 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.961459 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 12:15:54.962231 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.958471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:37765 (local address 127.7.39.2:36229)
0811 12:15:54.958628 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.958634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.958650 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:15:54.958695 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.958700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.958751 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.958837 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.960956 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.960961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.960966 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.960986 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.961025 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.961042 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.961195 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.961899 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.961964 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 12:15:54.962406 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.958962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:45129 (local address 127.7.39.1:42925)
0811 12:15:54.959124 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:54.959129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:54.959657 (+ 528us) server_negotiation.cc:407] Connection header received
0811 12:15:54.959880 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:54.959886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:54.959963 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:54.960033 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:54.961514 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:54.961519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:54.961523 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:54.961542 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:54.961585 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:54.961599 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:54.961692 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:54.962061 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 12:15:54.962121 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:15:54.963117 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:54.958817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.7.39.3:45129)
0811 12:15:54.959569 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:54.959585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:54.959732 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:54.960254 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:54.960259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:54.960283 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:15:54.961102 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:15:54.961107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:15:54.961126 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:15:54.961206 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:15:54.961923 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:15:54.961927 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:15:54.961931 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 12:15:54.962621 (+ 690us) client_negotiation.cc:240] Negotiation successful
0811 12:15:54.962678 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":635,"threads_started":1}
I20260811 12:15:54.963265 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592de7b77985441b9167bd442e5d1f40" candidate_uuid: "7f10c649ff4b4a8a98a36434ebad6067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86dfadd00ba46bd89a26346dcb48124" is_pre_election: true
I20260811 12:15:54.963965 14999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592de7b77985441b9167bd442e5d1f40" candidate_uuid: "7f10c649ff4b4a8a98a36434ebad6067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f3a652f2bc402fbdc540ac8002410e" is_pre_election: true
I20260811 12:15:54.964160 14999 raft_consensus.cc:2454] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f10c649ff4b4a8a98a36434ebad6067 in term 0.
I20260811 12:15:54.964552 15495 raft_consensus.cc:504] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.964656 15294 leader_election.cc:302] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21f3a652f2bc402fbdc540ac8002410e, 7f10c649ff4b4a8a98a36434ebad6067; no voters:
I20260811 12:15:54.965130 15498 raft_consensus.cc:2781] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:15:54.965242 15196 raft_consensus.cc:2379] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f10c649ff4b4a8a98a36434ebad6067 in current term 1: Already voted for candidate d86dfadd00ba46bd89a26346dcb48124 in this term.
I20260811 12:15:54.965338 15498 raft_consensus.cc:482] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:15:54.965447 15498 raft_consensus.cc:3037] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:54.965502 15495 leader_election.cc:288] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [CANDIDATE]: Term 1 election: Requested vote from peers 7f10c649ff4b4a8a98a36434ebad6067 (127.7.39.3:43535), 21f3a652f2bc402fbdc540ac8002410e (127.7.39.1:42925)
I20260811 12:15:54.965777 15398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592de7b77985441b9167bd442e5d1f40" candidate_uuid: "d86dfadd00ba46bd89a26346dcb48124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f10c649ff4b4a8a98a36434ebad6067"
I20260811 12:15:54.967331 14999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592de7b77985441b9167bd442e5d1f40" candidate_uuid: "d86dfadd00ba46bd89a26346dcb48124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f3a652f2bc402fbdc540ac8002410e"
I20260811 12:15:54.967581 14999 raft_consensus.cc:3037] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:54.968462 15498 raft_consensus.cc:504] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.968876 15398 raft_consensus.cc:2379] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d86dfadd00ba46bd89a26346dcb48124 in current term 1: Already voted for candidate 7f10c649ff4b4a8a98a36434ebad6067 in this term.
I20260811 12:15:54.969236 15498 leader_election.cc:288] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [CANDIDATE]: Term 1 election: Requested vote from peers d86dfadd00ba46bd89a26346dcb48124 (127.7.39.2:36229), 21f3a652f2bc402fbdc540ac8002410e (127.7.39.1:42925)
I20260811 12:15:54.969897 14998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592de7b77985441b9167bd442e5d1f40" candidate_uuid: "7f10c649ff4b4a8a98a36434ebad6067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f3a652f2bc402fbdc540ac8002410e"
I20260811 12:15:54.970574 14999 raft_consensus.cc:2454] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86dfadd00ba46bd89a26346dcb48124 in term 1.
I20260811 12:15:54.970525 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592de7b77985441b9167bd442e5d1f40" candidate_uuid: "7f10c649ff4b4a8a98a36434ebad6067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86dfadd00ba46bd89a26346dcb48124"
I20260811 12:15:54.970798 15196 raft_consensus.cc:2379] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f10c649ff4b4a8a98a36434ebad6067 in current term 1: Already voted for candidate d86dfadd00ba46bd89a26346dcb48124 in this term.
I20260811 12:15:54.971040 15093 leader_election.cc:302] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21f3a652f2bc402fbdc540ac8002410e, d86dfadd00ba46bd89a26346dcb48124; no voters: 7f10c649ff4b4a8a98a36434ebad6067
I20260811 12:15:54.971167 15294 leader_election.cc:302] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f10c649ff4b4a8a98a36434ebad6067; no voters: 21f3a652f2bc402fbdc540ac8002410e, d86dfadd00ba46bd89a26346dcb48124
I20260811 12:15:54.971380 15495 raft_consensus.cc:2781] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:54.971464 15498 raft_consensus.cc:2726] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 12:15:54.971853 15495 raft_consensus.cc:686] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 1 LEADER]: Becoming Leader. State: Replica: d86dfadd00ba46bd89a26346dcb48124, State: Running, Role: LEADER
I20260811 12:15:54.972322 15495 consensus_queue.cc:237] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } }
I20260811 12:15:54.977080 14778 catalog_manager.cc:5495] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 reported cstate change: term changed from 0 to 1, leader changed from <none> to d86dfadd00ba46bd89a26346dcb48124 (127.7.39.2). New cstate: current_term: 1 leader_uuid: "d86dfadd00ba46bd89a26346dcb48124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86dfadd00ba46bd89a26346dcb48124" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260811 12:15:55.002209 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.000197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.7.39.2:36229)
0811 12:15:55.000374 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:55.000379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:55.000490 (+ 111us) server_negotiation.cc:407] Connection header received
0811 12:15:55.000701 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:55.000705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:55.000811 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:55.000892 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:55.001712 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:55.001716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:55.001719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:55.001735 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:55.001767 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:55.001778 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:55.001879 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:55.001940 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 12:15:55.001991 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:15:55.010854 14999 raft_consensus.cc:1261] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 1 FOLLOWER]: Refusing update from remote peer d86dfadd00ba46bd89a26346dcb48124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:55.010874 15398 raft_consensus.cc:1261] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 1 FOLLOWER]: Refusing update from remote peer d86dfadd00ba46bd89a26346dcb48124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:15:55.011716 15495 consensus_queue.cc:1035] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21f3a652f2bc402fbdc540ac8002410e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:55.012009 15505 consensus_queue.cc:1035] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f10c649ff4b4a8a98a36434ebad6067" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:15:55.023782 15511 mvcc.cc:205] Tried to move back new op lower bound from 7317301473314467840 to 7317301473170878464. Current Snapshot: MvccSnapshot[applied={T|T < 7317301473314467840}]
I20260811 12:15:55.026697 15513 mvcc.cc:205] Tried to move back new op lower bound from 7317301473314467840 to 7317301473170878464. Current Snapshot: MvccSnapshot[applied={T|T < 7317301473314467840}]
I20260811 12:15:55.038913 14776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:55.039162 14776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:15:55.043162 14776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed01b34f12184a05bcba5c784c35ec6f: Sending DeleteTablet for 3 replicas of tablet 592de7b77985441b9167bd442e5d1f40
I20260811 12:15:55.044544 15369 tablet_service.cc:1514] Processing DeleteTablet for tablet 592de7b77985441b9167bd442e5d1f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:55 UTC) from {username='slave'} at 127.0.0.1:50704
I20260811 12:15:55.044584 15172 tablet_service.cc:1514] Processing DeleteTablet for tablet 592de7b77985441b9167bd442e5d1f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:55 UTC) from {username='slave'} at 127.0.0.1:59848
I20260811 12:15:55.044670 14960 tablet_service.cc:1514] Processing DeleteTablet for tablet 592de7b77985441b9167bd442e5d1f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:15:55 UTC) from {username='slave'} at 127.0.0.1:42380
I20260811 12:15:55.046986 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 14872
I20260811 12:15:55.057085 15526 tablet_replica.cc:330] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: stopping tablet replica
I20260811 12:15:55.057746 15526 raft_consensus.cc:2229] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:55.057802 15529 tablet_replica.cc:330] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: stopping tablet replica
I20260811 12:15:55.057919 15527 tablet_replica.cc:330] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: stopping tablet replica
I20260811 12:15:55.058306 15529 raft_consensus.cc:2229] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:15:55.058327 15527 raft_consensus.cc:2229] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:15:55.058573 15526 raft_consensus.cc:2258] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:55.058817 15527 raft_consensus.cc:2258] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:55.058952 15529 raft_consensus.cc:2258] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:15:55.060319 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.048931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.7.39.1:42925)
0811 12:15:55.049136 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:55.049143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:55.056235 (+ 7092us) server_negotiation.cc:407] Connection header received
0811 12:15:55.056570 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:55.056579 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:55.056708 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:55.056811 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:55.059497 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:55.059503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:55.059508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:55.059531 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:55.059570 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:55.059587 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:55.059690 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:55.059957 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 12:15:55.060024 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:15:55.061522 15062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:55.061620 15527 ts_tablet_manager.cc:1905] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:55.062762 15529 ts_tablet_manager.cc:1905] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:55.069492 15527 ts_tablet_manager.cc:1918] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:15:55.069693 15527 log.cc:1196] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/592de7b77985441b9167bd442e5d1f40
I20260811 12:15:55.070266 15527 ts_tablet_manager.cc:1939] T 592de7b77985441b9167bd442e5d1f40 P 7f10c649ff4b4a8a98a36434ebad6067: Deleting consensus metadata
I20260811 12:15:55.070914 15529 ts_tablet_manager.cc:1918] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:15:55.071120 15529 log.cc:1196] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/592de7b77985441b9167bd442e5d1f40
I20260811 12:15:55.071676 15529 ts_tablet_manager.cc:1939] T 592de7b77985441b9167bd442e5d1f40 P d86dfadd00ba46bd89a26346dcb48124: Deleting consensus metadata
I20260811 12:15:55.072525 14762 catalog_manager.cc:4841] TS 7f10c649ff4b4a8a98a36434ebad6067 (127.7.39.3:43535): tablet 592de7b77985441b9167bd442e5d1f40 (table test-table [id=efb4d73fec584b168348f7224ad7a7eb]) successfully deleted
I20260811 12:15:55.073477 14761 catalog_manager.cc:4841] TS d86dfadd00ba46bd89a26346dcb48124 (127.7.39.2:36229): tablet 592de7b77985441b9167bd442e5d1f40 (table test-table [id=efb4d73fec584b168348f7224ad7a7eb]) successfully deleted
I20260811 12:15:55.069743 15526 ts_tablet_manager.cc:1905] T 592de7b77985441b9167bd442e5d1f40 P 21f3a652f2bc402fbdc540ac8002410e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:15:55.178158 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 14872
I20260811 12:15:55.202298 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 15073
W20260811 12:15:55.202401 14761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.39.1:42925 (error 108)
I20260811 12:15:55.205027 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.203888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.0.0.1:42400)
0811 12:15:55.204674 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:55.204807 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":516,"threads_started":1}
W20260811 12:15:55.205500 14761 catalog_manager.cc:4584] TS 21f3a652f2bc402fbdc540ac8002410e (127.7.39.1:42925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 592de7b77985441b9167bd442e5d1f40: Network error: Client connection negotiation failed: client connection to 127.7.39.1:42925: connect: Connection refused (error 111)
I20260811 12:15:55.205615 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.203448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59862 (local address 127.7.39.2:36229)
0811 12:15:55.203640 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:55.203646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:55.203666 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:15:55.203740 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:55.203746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:55.203817 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:55.203922 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:55.204974 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:55.204978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:55.204981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:55.204996 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:55.205028 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:55.205040 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:55.205202 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:55.205282 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 12:15:55.205343 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:15:55.206404 15255 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:55.259105 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.258522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.0.0.1:42412)
0811 12:15:55.258777 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:55.258927 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 12:15:55.260355 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.259915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.0.0.1:42424)
0811 12:15:55.260076 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:55.260175 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 12:15:55.319542 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 15073
I20260811 12:15:55.330835 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.330338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.0.0.1:42432)
0811 12:15:55.330578 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:55.330686 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 12:15:55.331907 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.331504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.0.0.1:42448)
0811 12:15:55.331677 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:55.331763 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:15:55.347429 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 15267
I20260811 12:15:55.350199 15485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.348206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.7.39.3:43535)
0811 12:15:55.348363 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:55.348367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:55.348417 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:15:55.348572 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:55.348576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:55.348709 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:55.348801 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:55.349726 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:55.349730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:55.349733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:55.349746 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:55.349777 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:55.349788 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:55.349893 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:55.349928 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 12:15:55.349971 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:15:55.350883 15450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:55.412842 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.412358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.0.0.1:42460)
0811 12:15:55.412601 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:55.412695 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 12:15:55.414057 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:55.413600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42925 (local address 127.0.0.1:42470)
0811 12:15:55.413802 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:55.413892 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 12:15:55.439599 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 15267
I20260811 12:15:55.461277 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 14734
I20260811 12:15:55.462513 14825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:55.570753 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 14734
2026-08-11T12:15:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2946 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 12:15:55.615803 7324 mini_oidc.cc:152] Starting JWKS server
I20260811 12:15:55.619076 7324 webserver.cc:460] Webserver started at https://127.0.0.1:36425/ using document root <none> and password file <none>
I20260811 12:15:55.619464 7324 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:15:55.620194 7324 webserver.cc:460] Webserver started at http://127.0.0.1:39643/ using document root <none> and password file <none>
2026-08-11T12:15:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:55Z Disabled control of system clock
I20260811 12:15:55.654901 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35623
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:35623
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36425/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/testchainca.cert with env {}
W20260811 12:15:55.990757 15546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:55.991091 15546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:55.991225 15546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:55.998566 15546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:55.998715 15546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:55.998781 15546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:55.998860 15546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:55.998922 15546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:15:55.999012 15546 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:15:55.999070 15546 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36425/jwks.json
I20260811 12:15:56.009644 15546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:35623
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36425/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:56.011053 15546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:56.012425 15546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:56.024518 15553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:56.024551 15556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:56.024608 15554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:56.025395 15546 server_base.cc:1034] running on GCE node
I20260811 12:15:56.026194 15546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:56.027585 15546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:56.028802 15546 hybrid_clock.cc:648] HybridClock initialized: now 1786450556028746 us; error 50 us; skew 500 ppm
I20260811 12:15:56.029181 15546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:15:56.031390 15546 webserver.cc:460] Webserver started at http://127.7.39.62:34261/ using document root <none> and password file <none>
I20260811 12:15:56.032084 15546 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:56.032199 15546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:56.032577 15546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:56.035630 15546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "0e3479fc82814a188a96d05fa63fea73"
format_stamp: "Formatted at 2026-08-11 12:15:56 on dist-test-slave-rmhg"
server_key: "79157452b02ce19ae55c14edbf21a5fa"
server_key_iv: "b37674c569160cd2e4c8d3b404ad4c5e"
server_key_version: "encryptionkey@0"
I20260811 12:15:56.036427 15546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "0e3479fc82814a188a96d05fa63fea73"
format_stamp: "Formatted at 2026-08-11 12:15:56 on dist-test-slave-rmhg"
server_key: "79157452b02ce19ae55c14edbf21a5fa"
server_key_iv: "b37674c569160cd2e4c8d3b404ad4c5e"
server_key_version: "encryptionkey@0"
I20260811 12:15:56.041795 15546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 12:15:56.045497 15569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:56.046921 15546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:15:56.047096 15546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "0e3479fc82814a188a96d05fa63fea73"
format_stamp: "Formatted at 2026-08-11 12:15:56 on dist-test-slave-rmhg"
server_key: "79157452b02ce19ae55c14edbf21a5fa"
server_key_iv: "b37674c569160cd2e4c8d3b404ad4c5e"
server_key_version: "encryptionkey@0"
I20260811 12:15:56.047305 15546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:56.069821 15546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:56.070735 15546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:56.071036 15546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:56.089120 15546 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:35623
I20260811 12:15:56.089144 15642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:35623 every 8 connection(s)
I20260811 12:15:56.090596 15546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:56.094574 15643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:56.096316 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 15546
I20260811 12:15:56.096544 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:56.097208 7324 external_mini_cluster.cc:1428] Setting key 533f5e789a06cbb0cf763ec7950b8fd0
I20260811 12:15:56.107235 15643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73: Bootstrap starting.
I20260811 12:15:56.110949 15643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:56.111886 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:56.099404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.7.39.62:35623)
0811 12:15:56.100457 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:56.100469 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:56.100515 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:15:56.102326 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:56.102354 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:56.102853 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:56.103210 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:56.103862 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:56.105321 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:56.107219 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:56.107961 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:56.108104 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:56.109448 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:56.109456 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:56.109471 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:56.110047 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:56.110144 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:56.110170 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:56.110256 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:56.110482 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 12:15:56.110826 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":339,"threads_started":1}
I20260811 12:15:56.112046 15643 log.cc:824] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:56.115411 15643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73: No bootstrap required, opened a new log
I20260811 12:15:56.116147 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:15:56.124262 15643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3479fc82814a188a96d05fa63fea73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35623 } }
I20260811 12:15:56.124678 15643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:56.124766 15643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3479fc82814a188a96d05fa63fea73, State: Initialized, Role: FOLLOWER
I20260811 12:15:56.125322 15643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3479fc82814a188a96d05fa63fea73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35623 } }
I20260811 12:15:56.125542 15643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:56.125655 15643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:56.125783 15643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:56.127969 15643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3479fc82814a188a96d05fa63fea73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35623 } }
I20260811 12:15:56.128490 15643 leader_election.cc:302] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e3479fc82814a188a96d05fa63fea73; no voters:
I20260811 12:15:56.129143 15643 leader_election.cc:288] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:56.129364 15652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:56.130427 15652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [term 1 LEADER]: Becoming Leader. State: Replica: 0e3479fc82814a188a96d05fa63fea73, State: Running, Role: LEADER
I20260811 12:15:56.130959 15652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3479fc82814a188a96d05fa63fea73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35623 } }
I20260811 12:15:56.131467 15643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:56.134060 15655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e3479fc82814a188a96d05fa63fea73. Latest consensus state: current_term: 1 leader_uuid: "0e3479fc82814a188a96d05fa63fea73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3479fc82814a188a96d05fa63fea73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35623 } } }
I20260811 12:15:56.133991 15654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e3479fc82814a188a96d05fa63fea73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e3479fc82814a188a96d05fa63fea73" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35623 } } }
I20260811 12:15:56.134469 15655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:56.134469 15654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73 [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:56.135396 15664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:56.141009 15664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:56.156396 15664 catalog_manager.cc:1347] Generated new cluster ID: 16e9a2b2f6134ace8e9f92af2e7a35a1
I20260811 12:15:56.156553 15664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:56.183423 15664 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:56.184845 15664 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:56.202522 15664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e3479fc82814a188a96d05fa63fea73: Generated new TSK 0
I20260811 12:15:56.203460 15664 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 12:15:56.247010 15690 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 12:15:56.247248 15690 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:15:56.248553 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:56.225146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.7.39.62:35623)
0811 12:15:56.225379 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:56.225386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:56.225755 (+ 369us) server_negotiation.cc:407] Connection header received
0811 12:15:56.226135 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:56.226142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:56.226245 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:56.226432 (+ 187us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:15:56.226975 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:56.228145 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:56.229664 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:56.230388 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:56.230535 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:56.231048 (+ 513us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:15:56.247818 (+ 16770us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 12:15:56.248224 (+ 406us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45948: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 12:15:56.248703 15648 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45948: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 12:15:56.250288 15689 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:15:56.224940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:35623 (local address 127.0.0.1:45948)
0811 12:15:56.225643 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:56.225662 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:56.225832 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:56.226441 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:56.226445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:56.226456 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:15:56.226797 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:56.226806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:56.228282 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:56.228285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:56.229507 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:56.229520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:56.230501 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:56.230560 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:56.230687 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:56.230756 (+ 69us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:15:56.248136 (+ 17380us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 12:15:56.248455 (+ 319us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.39.62:35623: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":358,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.39.62:35623: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36425/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 12:15:56.252118 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 15546
I20260811 12:15:56.253386 15635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:56.351528 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 15546
2026-08-11T12:15:56Z chronyd exiting
I20260811 12:15:56.408798 7324 test_util.cc:175] -----------------------------------------------
I20260811 12:15:56.408957 7324 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450508543501-7324-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (798 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 12:15:56.413017 7324 mini_oidc.cc:152] Starting JWKS server
I20260811 12:15:56.414985 7324 webserver.cc:460] Webserver started at https://127.0.0.1:38701/ using document root <none> and password file <none>
I20260811 12:15:56.415218 7324 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:15:56.415890 7324 webserver.cc:460] Webserver started at http://127.0.0.1:35659/ using document root <none> and password file <none>
2026-08-11T12:15:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:56Z Disabled control of system clock
I20260811 12:15:56.451838 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:37285
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:37285
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38701/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/testchainca.cert with env {}
W20260811 12:15:56.794370 15700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:56.794706 15700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:56.794847 15700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:56.802253 15700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:56.802385 15700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:56.802428 15700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:56.802508 15700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:56.802567 15700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:15:56.802634 15700 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:15:56.802696 15700 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38701/jwks.json
I20260811 12:15:56.813109 15700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:37285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:37285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38701/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:56.814600 15700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:56.816074 15700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:56.828297 15706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:56.828341 15714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:56.828370 15710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:56.829069 15700 server_base.cc:1034] running on GCE node
I20260811 12:15:56.830039 15700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:56.831564 15700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:56.832783 15700 hybrid_clock.cc:648] HybridClock initialized: now 1786450556832730 us; error 48 us; skew 500 ppm
I20260811 12:15:56.833143 15700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:15:56.835625 15700 webserver.cc:460] Webserver started at http://127.7.39.62:43141/ using document root <none> and password file <none>
I20260811 12:15:56.836361 15700 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:56.836475 15700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:56.836820 15700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:56.839816 15700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "354a34743af342e3992e2804bb3cc15b"
format_stamp: "Formatted at 2026-08-11 12:15:56 on dist-test-slave-rmhg"
server_key: "5c83e064beb982a18f30d5c5d55b794b"
server_key_iv: "c74daff13082cd77e6a2855f247701ff"
server_key_version: "encryptionkey@0"
I20260811 12:15:56.840647 15700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "354a34743af342e3992e2804bb3cc15b"
format_stamp: "Formatted at 2026-08-11 12:15:56 on dist-test-slave-rmhg"
server_key: "5c83e064beb982a18f30d5c5d55b794b"
server_key_iv: "c74daff13082cd77e6a2855f247701ff"
server_key_version: "encryptionkey@0"
I20260811 12:15:56.846056 15700 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:15:56.849597 15720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:56.850975 15700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:15:56.851150 15700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "354a34743af342e3992e2804bb3cc15b"
format_stamp: "Formatted at 2026-08-11 12:15:56 on dist-test-slave-rmhg"
server_key: "5c83e064beb982a18f30d5c5d55b794b"
server_key_iv: "c74daff13082cd77e6a2855f247701ff"
server_key_version: "encryptionkey@0"
I20260811 12:15:56.851359 15700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:56.872324 15700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:56.873291 15700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:56.873580 15700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:56.891532 15700 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:37285
I20260811 12:15:56.891579 15802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:37285 every 8 connection(s)
I20260811 12:15:56.892930 15700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:56.895466 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 15700
I20260811 12:15:56.895686 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:56.896535 7324 external_mini_cluster.cc:1428] Setting key 76a9ca4e9493a88ba51affefff715361
I20260811 12:15:56.897542 15804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:56.911484 15804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b: Bootstrap starting.
I20260811 12:15:56.911518 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:56.898583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.7.39.62:37285)
0811 12:15:56.899602 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:56.899614 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:56.899658 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:15:56.902047 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:56.902072 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:56.902597 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:56.902951 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:56.903622 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:56.905015 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:56.906843 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:56.907563 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:56.907700 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:56.908959 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:56.908968 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:56.908983 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:56.909617 (+ 634us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:56.909711 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:56.909733 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:56.909819 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:56.910154 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 12:15:56.910513 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":372,"threads_started":1}
I20260811 12:15:56.914781 15804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:56.915392 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:15:56.915977 15804 log.cc:824] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:56.919013 15804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b: No bootstrap required, opened a new log
I20260811 12:15:56.926156 15804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354a34743af342e3992e2804bb3cc15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37285 } }
I20260811 12:15:56.926517 15804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:56.926622 15804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354a34743af342e3992e2804bb3cc15b, State: Initialized, Role: FOLLOWER
I20260811 12:15:56.927172 15804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354a34743af342e3992e2804bb3cc15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37285 } }
I20260811 12:15:56.927378 15804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:56.927511 15804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:56.927671 15804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:56.929899 15804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354a34743af342e3992e2804bb3cc15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37285 } }
I20260811 12:15:56.930411 15804 leader_election.cc:302] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 354a34743af342e3992e2804bb3cc15b; no voters:
I20260811 12:15:56.931030 15804 leader_election.cc:288] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:56.931141 15811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:56.932108 15811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [term 1 LEADER]: Becoming Leader. State: Replica: 354a34743af342e3992e2804bb3cc15b, State: Running, Role: LEADER
I20260811 12:15:56.932638 15811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354a34743af342e3992e2804bb3cc15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37285 } }
I20260811 12:15:56.933110 15804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:56.935302 15816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 354a34743af342e3992e2804bb3cc15b. Latest consensus state: current_term: 1 leader_uuid: "354a34743af342e3992e2804bb3cc15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354a34743af342e3992e2804bb3cc15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37285 } } }
I20260811 12:15:56.935391 15815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "354a34743af342e3992e2804bb3cc15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354a34743af342e3992e2804bb3cc15b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37285 } } }
I20260811 12:15:56.935856 15816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:56.935855 15815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:56.937611 15821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:56.942792 15821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:56.951956 15821 catalog_manager.cc:1347] Generated new cluster ID: 9142d903bb494f28ac1067e239cb930c
I20260811 12:15:56.952072 15821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:56.989336 15821 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:56.990686 15821 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:57.000756 15821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 354a34743af342e3992e2804bb3cc15b: Generated new TSK 0
I20260811 12:15:57.001706 15821 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 12:15:57.046605 15848 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 12:15:57.046746 15848 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:15:57.047924 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:57.020807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.7.39.62:37285)
0811 12:15:57.021048 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:57.021054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:57.021457 (+ 403us) server_negotiation.cc:407] Connection header received
0811 12:15:57.021770 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:57.021774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:57.021858 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:57.022027 (+ 169us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:15:57.022555 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.023331 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.024768 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.025253 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.025374 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:57.025887 (+ 513us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:15:57.047132 (+ 21245us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 12:15:57.047555 (+ 423us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36068: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":95}
W20260811 12:15:57.048067 15808 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36068: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 12:15:57.048144 15846 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:15:57.020601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37285 (local address 127.0.0.1:36068)
0811 12:15:57.021335 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:57.021356 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:57.021538 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:57.022069 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:57.022075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:57.022088 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:15:57.022383 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:57.022393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.023485 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.023490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:57.024612 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:57.024625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.025371 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.025425 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:57.025587 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:57.025624 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:15:57.047419 (+ 21795us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 12:15:57.047721 (+ 302us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.39.62:37285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":369,"threads_started":1}
I20260811 12:15:57.049858 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 15700
I20260811 12:15:57.051000 15797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:57.147064 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 15700
2026-08-11T12:15:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (796 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 12:15:57.209390 7324 mini_oidc.cc:152] Starting JWKS server
I20260811 12:15:57.211339 7324 webserver.cc:460] Webserver started at https://127.0.0.1:40851/ using document root <none> and password file <none>
I20260811 12:15:57.211584 7324 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:15:57.212229 7324 webserver.cc:460] Webserver started at http://127.0.0.1:43727/ using document root <none> and password file <none>
2026-08-11T12:15:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:57Z Disabled control of system clock
I20260811 12:15:57.247160 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34691
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:43425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34691
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40851/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 12:15:57.588587 15857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:57.588979 15857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:57.589121 15857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:57.596544 15857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:57.596668 15857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:57.596732 15857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:57.596813 15857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:57.596879 15857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:15:57.596968 15857 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:15:57.597026 15857 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40851/jwks.json
I20260811 12:15:57.606705 15857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34691
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40851/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:57.608089 15857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:57.610046 15857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:57.622035 15865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:57.622131 15869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:57.622067 15867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:57.623090 15857 server_base.cc:1034] running on GCE node
I20260811 12:15:57.623895 15857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:57.625371 15857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:57.626566 15857 hybrid_clock.cc:648] HybridClock initialized: now 1786450557626510 us; error 55 us; skew 500 ppm
I20260811 12:15:57.626906 15857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:15:57.629217 15857 webserver.cc:460] Webserver started at http://127.7.39.62:40791/ using document root <none> and password file <none>
I20260811 12:15:57.629963 15857 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:57.630074 15857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:57.630419 15857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:57.633543 15857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "4453890610cf411cbbad6a5627ecbc2e"
format_stamp: "Formatted at 2026-08-11 12:15:57 on dist-test-slave-rmhg"
server_key: "655e1268f84e7d9aca1968ca8d4daca7"
server_key_iv: "3361f54989fe6dd49a572bd90eaf6612"
server_key_version: "encryptionkey@0"
I20260811 12:15:57.634373 15857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "4453890610cf411cbbad6a5627ecbc2e"
format_stamp: "Formatted at 2026-08-11 12:15:57 on dist-test-slave-rmhg"
server_key: "655e1268f84e7d9aca1968ca8d4daca7"
server_key_iv: "3361f54989fe6dd49a572bd90eaf6612"
server_key_version: "encryptionkey@0"
I20260811 12:15:57.639683 15857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:15:57.643249 15878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:57.644768 15857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:15:57.644959 15857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "4453890610cf411cbbad6a5627ecbc2e"
format_stamp: "Formatted at 2026-08-11 12:15:57 on dist-test-slave-rmhg"
server_key: "655e1268f84e7d9aca1968ca8d4daca7"
server_key_iv: "3361f54989fe6dd49a572bd90eaf6612"
server_key_version: "encryptionkey@0"
I20260811 12:15:57.645283 15857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:57.664765 15857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:57.665676 15857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:57.665925 15857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:57.683874 15857 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:34691
I20260811 12:15:57.683894 15966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:34691 every 8 connection(s)
I20260811 12:15:57.685284 15857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:57.689006 15971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:57.691058 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 15857
I20260811 12:15:57.691277 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:57.692097 7324 external_mini_cluster.cc:1428] Setting key 4f743842d26457b0e03342e0a767868d
I20260811 12:15:57.701411 15971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e: Bootstrap starting.
I20260811 12:15:57.705008 15971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:57.706121 15971 log.cc:824] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:57.706214 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:57.694103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.7.39.62:34691)
0811 12:15:57.695149 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:57.695160 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:57.695207 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:15:57.696843 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:57.696868 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:57.697388 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:57.697756 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:57.698352 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.699739 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.701604 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.702282 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.702426 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:57.703662 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:57.703671 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:57.703686 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:57.704228 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:57.704319 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:57.704341 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:57.704448 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:57.704659 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 12:15:57.705183 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":349,"threads_started":1}
I20260811 12:15:57.708981 15971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e: No bootstrap required, opened a new log
I20260811 12:15:57.710067 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:15:57.716671 15971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4453890610cf411cbbad6a5627ecbc2e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34691 } }
I20260811 12:15:57.717025 15971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:57.717144 15971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4453890610cf411cbbad6a5627ecbc2e, State: Initialized, Role: FOLLOWER
I20260811 12:15:57.717756 15971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4453890610cf411cbbad6a5627ecbc2e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34691 } }
I20260811 12:15:57.717999 15971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:57.718133 15971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:57.718286 15971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:57.720709 15971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4453890610cf411cbbad6a5627ecbc2e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34691 } }
I20260811 12:15:57.721232 15971 leader_election.cc:302] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4453890610cf411cbbad6a5627ecbc2e; no voters:
I20260811 12:15:57.721794 15971 leader_election.cc:288] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:57.721926 15977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:57.722559 15977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [term 1 LEADER]: Becoming Leader. State: Replica: 4453890610cf411cbbad6a5627ecbc2e, State: Running, Role: LEADER
I20260811 12:15:57.723060 15977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4453890610cf411cbbad6a5627ecbc2e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34691 } }
I20260811 12:15:57.723835 15971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:57.725988 15979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4453890610cf411cbbad6a5627ecbc2e. Latest consensus state: current_term: 1 leader_uuid: "4453890610cf411cbbad6a5627ecbc2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4453890610cf411cbbad6a5627ecbc2e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34691 } } }
I20260811 12:15:57.725997 15978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4453890610cf411cbbad6a5627ecbc2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4453890610cf411cbbad6a5627ecbc2e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34691 } } }
I20260811 12:15:57.726425 15978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:57.726425 15979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:57.730464 15988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:57.735881 15988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:57.744750 15988 catalog_manager.cc:1347] Generated new cluster ID: a56a103ea18444d3af71d6d3c16eea36
I20260811 12:15:57.744858 15988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:57.756352 15988 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:57.757669 15988 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:57.766242 15988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4453890610cf411cbbad6a5627ecbc2e: Generated new TSK 0
I20260811 12:15:57.767086 15988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:15:57.792900 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:57.785489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.7.39.62:34691)
0811 12:15:57.785667 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:57.785675 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:57.786105 (+ 430us) server_negotiation.cc:407] Connection header received
0811 12:15:57.786338 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:57.786342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:57.786418 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:57.786564 (+ 146us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:15:57.787054 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.787847 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.789255 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.789717 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.789834 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:57.790363 (+ 529us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:15:57.792260 (+ 1897us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 12:15:57.792581 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44884: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 12:15:57.793045 15975 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44884: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 12:15:57.793092 16012 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:15:57.785394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34691 (local address 127.0.0.1:44884)
0811 12:15:57.785992 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:57.786009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:57.786157 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:57.786594 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:57.786599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:57.786610 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:15:57.786894 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:57.786904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.787999 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.788003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:57.789095 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:57.789106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:57.789838 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:57.789922 (+ 84us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:57.790052 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:57.790086 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:15:57.792459 (+ 2373us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 12:15:57.792704 (+ 245us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.39.62:34691: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40851/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":337,"threads_started":1}
I20260811 12:15:57.794588 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 15857
I20260811 12:15:57.795794 15959 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 12:15:57.805016 16013 webserver.cc:533] Webserver: sslize failed:
E20260811 12:15:57.805127 16013 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:15:57.888659 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 15857
2026-08-11T12:15:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (739 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T12:15:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:15:57Z Disabled control of system clock
I20260811 12:15:57.983100 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:37233
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:41211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:37233
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/testchainca.cert with env {}
W20260811 12:15:58.324038 16021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:15:58.324373 16021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:15:58.324509 16021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:15:58.332074 16021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:15:58.332211 16021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:15:58.332254 16021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:15:58.332335 16021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:15:58.332388 16021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:15:58.332475 16021 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:15:58.332522 16021 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 12:15:58.343063 16021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:41211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:37233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:37233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:15:58.344556 16021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:15:58.346001 16021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:15:58.358402 16033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:58.358438 16034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:15:58.358372 16036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:15:58.359107 16021 server_base.cc:1034] running on GCE node
I20260811 12:15:58.360049 16021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:15:58.361624 16021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:15:58.362849 16021 hybrid_clock.cc:648] HybridClock initialized: now 1786450558362798 us; error 44 us; skew 500 ppm
I20260811 12:15:58.363240 16021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:15:58.365823 16021 webserver.cc:460] Webserver started at http://127.7.39.62:37123/ using document root <none> and password file <none>
I20260811 12:15:58.366629 16021 fs_manager.cc:359] Metadata directory not provided
I20260811 12:15:58.366739 16021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:15:58.367069 16021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:15:58.370256 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "d2b67106865746db8fdf255b911001bb"
format_stamp: "Formatted at 2026-08-11 12:15:58 on dist-test-slave-rmhg"
server_key: "b7588e2e2fa7357c174f4674e4856e2d"
server_key_iv: "f65611ad795240b17330ed94841335a3"
server_key_version: "encryptionkey@0"
I20260811 12:15:58.371109 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "d2b67106865746db8fdf255b911001bb"
format_stamp: "Formatted at 2026-08-11 12:15:58 on dist-test-slave-rmhg"
server_key: "b7588e2e2fa7357c174f4674e4856e2d"
server_key_iv: "f65611ad795240b17330ed94841335a3"
server_key_version: "encryptionkey@0"
I20260811 12:15:58.377183 16021 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 12:15:58.381536 16043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:15:58.383320 16021 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 12:15:58.383533 16021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "d2b67106865746db8fdf255b911001bb"
format_stamp: "Formatted at 2026-08-11 12:15:58 on dist-test-slave-rmhg"
server_key: "b7588e2e2fa7357c174f4674e4856e2d"
server_key_iv: "f65611ad795240b17330ed94841335a3"
server_key_version: "encryptionkey@0"
I20260811 12:15:58.383805 16021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:15:58.395056 16021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:15:58.395984 16021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:15:58.396276 16021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:15:58.414067 16021 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:37233
I20260811 12:15:58.414088 16134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:37233 every 8 connection(s)
I20260811 12:15:58.415496 16021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:15:58.416066 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 16021
I20260811 12:15:58.416262 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:15:58.417068 7324 external_mini_cluster.cc:1428] Setting key 9d72a404058d1f563d656c5eceaf4407
I20260811 12:15:58.420027 16136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:15:58.431521 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:58.419064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60390 (local address 127.7.39.62:37233)
0811 12:15:58.420071 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:58.420083 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:58.420129 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:15:58.421833 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:58.421858 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:58.422342 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:58.422721 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:15:58.423386 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:58.424789 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:58.426696 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:58.427407 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:58.427549 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:58.428869 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:15:58.428876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:15:58.428893 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:15:58.429564 (+ 671us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:15:58.429655 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:15:58.429687 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:15:58.429808 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:15:58.430051 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 12:15:58.430430 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":345,"threads_started":1}
I20260811 12:15:58.432786 16136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb: Bootstrap starting.
I20260811 12:15:58.435575 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:15:58.436419 16136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb: Neither blocks nor log segments found. Creating new log.
I20260811 12:15:58.437747 16136 log.cc:824] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:15:58.441049 16136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb: No bootstrap required, opened a new log
I20260811 12:15:58.448979 16136 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b67106865746db8fdf255b911001bb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37233 } }
I20260811 12:15:58.449465 16136 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:15:58.449594 16136 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b67106865746db8fdf255b911001bb, State: Initialized, Role: FOLLOWER
I20260811 12:15:58.450212 16136 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b67106865746db8fdf255b911001bb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37233 } }
I20260811 12:15:58.450436 16136 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:15:58.450546 16136 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:15:58.450690 16136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:15:58.452979 16136 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b67106865746db8fdf255b911001bb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37233 } }
I20260811 12:15:58.453560 16136 leader_election.cc:302] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2b67106865746db8fdf255b911001bb; no voters:
I20260811 12:15:58.454293 16136 leader_election.cc:288] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:15:58.454404 16145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:15:58.455204 16145 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [term 1 LEADER]: Becoming Leader. State: Replica: d2b67106865746db8fdf255b911001bb, State: Running, Role: LEADER
I20260811 12:15:58.455830 16145 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b67106865746db8fdf255b911001bb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37233 } }
I20260811 12:15:58.456602 16136 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:15:58.461066 16147 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2b67106865746db8fdf255b911001bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b67106865746db8fdf255b911001bb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37233 } } }
I20260811 12:15:58.461567 16147 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:58.460994 16146 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2b67106865746db8fdf255b911001bb. Latest consensus state: current_term: 1 leader_uuid: "d2b67106865746db8fdf255b911001bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b67106865746db8fdf255b911001bb" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37233 } } }
I20260811 12:15:58.461851 16146 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb [sys.catalog]: This master's current role is: LEADER
I20260811 12:15:58.462910 16160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:15:58.467516 16160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:15:58.475970 16160 catalog_manager.cc:1347] Generated new cluster ID: a3613c2c535144249d641ba678399fc9
I20260811 12:15:58.476110 16160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:15:58.490080 16160 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:15:58.491417 16160 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:15:58.500725 16160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2b67106865746db8fdf255b911001bb: Generated new TSK 0
I20260811 12:15:58.501686 16160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:15:58.636726 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:15:58.526153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60394 (local address 127.7.39.62:37233)
0811 12:15:58.526368 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:15:58.526376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:15:58.526827 (+ 451us) server_negotiation.cc:407] Connection header received
0811 12:15:58.527081 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:15:58.527087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:15:58.527205 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:15:58.527399 (+ 194us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:15:58.527936 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:58.529106 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:58.530671 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:15:58.531405 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:15:58.531543 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:58.532062 (+ 519us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:15:58.635933 (+103871us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 12:15:58.636402 (+ 469us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60394: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 12:15:58.636889 16142 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60394: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 12:15:58.636883 16182 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:15:58.525948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37233 (local address 127.0.0.1:60394)
0811 12:15:58.526710 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 12:15:58.526727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:15:58.526871 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:15:58.527422 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:15:58.527425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:15:58.527436 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:15:58.527768 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:58.527778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:58.529320 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:58.529325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:58.530509 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:15:58.530524 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:15:58.531540 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:15:58.531590 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:15:58.531725 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:15:58.531761 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:15:58.636279 (+104518us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 12:15:58.636548 (+ 269us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.39.62:37233: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":417,"threads_started":1}
I20260811 12:15:58.638528 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 16021
I20260811 12:15:58.639856 16128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:15:58.748754 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 16021
2026-08-11T12:15:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (840 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (424 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (428 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:15:59 dist-test-slave-rmhg krb5kdc[16208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:15:59 dist-test-slave-rmhg krb5kdc[16208](info): set up 2 sockets
Aug 11 12:15:59 dist-test-slave-rmhg krb5kdc[16208](info): commencing operation
krb5kdc: starting...
W20260811 12:16:00.843703 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:16:01 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:16:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:16:01.097071 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:44691
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:44691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:01.446501 16227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:01.446817 16227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:01.446915 16227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:01.454303 16227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:01.454432 16227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:01.454487 16227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:01.454543 16227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:01.454587 16227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:01.464830 16227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:44691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:44691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:01.466272 16227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:01.467597 16227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:01.479931 16236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:01.479867 16244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:01.479866 16237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:01.480875 16227 server_base.cc:1034] running on GCE node
I20260811 12:16:01.481822 16227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:01.483425 16227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:01.484680 16227 hybrid_clock.cc:648] HybridClock initialized: now 1786450561484615 us; error 53 us; skew 500 ppm
I20260811 12:16:01.485038 16227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:01 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450561, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:01.488184 16227 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:16:01.489825 16227 webserver.cc:460] Webserver started at http://127.7.39.62:38393/ using document root <none> and password file <none>
I20260811 12:16:01.490553 16227 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:01.490689 16227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:01.491025 16227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:01.494341 16227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "85ddc77d905b4e3383e822c4509171d4"
format_stamp: "Formatted at 2026-08-11 12:16:01 on dist-test-slave-rmhg"
server_key: "b7fcc89bf15f0dd28029d07012798b1a"
server_key_iv: "fd947aa0b53882ebbd5ea76d247baf7a"
server_key_version: "encryptionkey@0"
I20260811 12:16:01.495178 16227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "85ddc77d905b4e3383e822c4509171d4"
format_stamp: "Formatted at 2026-08-11 12:16:01 on dist-test-slave-rmhg"
server_key: "b7fcc89bf15f0dd28029d07012798b1a"
server_key_iv: "fd947aa0b53882ebbd5ea76d247baf7a"
server_key_version: "encryptionkey@0"
I20260811 12:16:01.500800 16227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:16:01.504534 16254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:01.505997 16227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:16:01.506176 16227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "85ddc77d905b4e3383e822c4509171d4"
format_stamp: "Formatted at 2026-08-11 12:16:01 on dist-test-slave-rmhg"
server_key: "b7fcc89bf15f0dd28029d07012798b1a"
server_key_iv: "fd947aa0b53882ebbd5ea76d247baf7a"
server_key_version: "encryptionkey@0"
I20260811 12:16:01.506369 16227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:01.526089 16227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:01.530627 16227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:01.530921 16227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:01.549209 16227 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:44691
I20260811 12:16:01.549244 16340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:44691 every 8 connection(s)
I20260811 12:16:01.550645 16227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:16:01.553385 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 16227
I20260811 12:16:01.553604 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:16:01.554443 7324 external_mini_cluster.cc:1428] Setting key 9dd6e2b1db7527f8aa03fa5a3853a130
I20260811 12:16:01.555402 16341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:16:01 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:01.570377 16341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4: Bootstrap starting.
I20260811 12:16:01.574753 16341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:01.576083 16341 log.cc:824] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:01.579941 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:01.556681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45236 (local address 127.7.39.62:44691)
0811 12:16:01.557554 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:01.557571 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:01.557627 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:16:01.559426 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:01.559459 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:01.560059 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:01.560547 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:01.562007 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:01.564111 (+ 2104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:01.565473 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:01.566537 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:01.566757 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:01.570932 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:01.570942 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:01.570966 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:01.571014 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:01.575022 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:01.575846 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:01.575856 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:01.575865 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:01.576103 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:01.576534 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:01.576540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:01.576543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:01.577488 (+ 945us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:01.577866 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:01.578345 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 12:16:01.578849 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":359,"threads_started":1}
I20260811 12:16:01.580668 16341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4: No bootstrap required, opened a new log
I20260811 12:16:01.588909 16341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ddc77d905b4e3383e822c4509171d4" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44691 } }
I20260811 12:16:01.589361 16341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:01.589453 16341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ddc77d905b4e3383e822c4509171d4, State: Initialized, Role: FOLLOWER
I20260811 12:16:01.590008 16341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ddc77d905b4e3383e822c4509171d4" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44691 } }
I20260811 12:16:01.590214 16341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:16:01.590315 16341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:16:01.590444 16341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:01.592787 16341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ddc77d905b4e3383e822c4509171d4" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44691 } }
I20260811 12:16:01.593328 16341 leader_election.cc:302] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85ddc77d905b4e3383e822c4509171d4; no voters:
I20260811 12:16:01.593889 16341 leader_election.cc:288] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:16:01.594019 16347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:01.594648 16347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [term 1 LEADER]: Becoming Leader. State: Replica: 85ddc77d905b4e3383e822c4509171d4, State: Running, Role: LEADER
I20260811 12:16:01.595177 16347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ddc77d905b4e3383e822c4509171d4" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44691 } }
I20260811 12:16:01.595906 16341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:01.597939 16350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85ddc77d905b4e3383e822c4509171d4. Latest consensus state: current_term: 1 leader_uuid: "85ddc77d905b4e3383e822c4509171d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ddc77d905b4e3383e822c4509171d4" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44691 } } }
I20260811 12:16:01.598412 16350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:01.598783 16349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85ddc77d905b4e3383e822c4509171d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ddc77d905b4e3383e822c4509171d4" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44691 } } }
I20260811 12:16:01.599131 16349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:01.599898 16359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:01.605006 16359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:01.614800 16359 catalog_manager.cc:1347] Generated new cluster ID: f370634c181d4f10ab34cb4bb82c185c
I20260811 12:16:01.614938 16359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:01.632318 16359 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:16:01.634325 16359 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:01.645233 16359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85ddc77d905b4e3383e822c4509171d4: Generated new TSK 0
I20260811 12:16:01.646185 16359 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:16:01.724880 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44691
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:02.068583 16382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:02.068882 16382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:02.068984 16382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:02.076606 16382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:02.076719 16382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:02.076833 16382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:02.087033 16382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:02.088351 16382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:02.089764 16382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:02.103343 16392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:02.103382 16398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:02.103305 16395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:02.104233 16382 server_base.cc:1034] running on GCE node
I20260811 12:16:02.104903 16382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:02.105980 16382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:02.107226 16382 hybrid_clock.cc:648] HybridClock initialized: now 1786450562107163 us; error 71 us; skew 500 ppm
I20260811 12:16:02.107585 16382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:02 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:02.110991 16382 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:16:02.112684 16382 webserver.cc:460] Webserver started at http://127.7.39.1:43879/ using document root <none> and password file <none>
I20260811 12:16:02.113474 16382 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:02.113603 16382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:02.113966 16382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:02.117050 16382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "39a84466928d4d77b7d698c014a4ea21"
format_stamp: "Formatted at 2026-08-11 12:16:02 on dist-test-slave-rmhg"
server_key: "620349b416dbe4db8ea43b2cba95278e"
server_key_iv: "9304178d49fde6302537cefe55e49031"
server_key_version: "encryptionkey@0"
I20260811 12:16:02.117928 16382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "39a84466928d4d77b7d698c014a4ea21"
format_stamp: "Formatted at 2026-08-11 12:16:02 on dist-test-slave-rmhg"
server_key: "620349b416dbe4db8ea43b2cba95278e"
server_key_iv: "9304178d49fde6302537cefe55e49031"
server_key_version: "encryptionkey@0"
I20260811 12:16:02.123349 16382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:16:02.127043 16408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:02.128458 16382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:16:02.128634 16382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "39a84466928d4d77b7d698c014a4ea21"
format_stamp: "Formatted at 2026-08-11 12:16:02 on dist-test-slave-rmhg"
server_key: "620349b416dbe4db8ea43b2cba95278e"
server_key_iv: "9304178d49fde6302537cefe55e49031"
server_key_version: "encryptionkey@0"
I20260811 12:16:02.128846 16382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:02.148059 16382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:02.152503 16382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:02.152798 16382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:02.153846 16382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:02.155275 16382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:02.155357 16382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:02.155483 16382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:02.155536 16382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:02.187006 16382 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:35487
I20260811 12:16:02.187052 16583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:35487 every 8 connection(s)
I20260811 12:16:02.188308 16382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:02.191448 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 16382
I20260811 12:16:02.191639 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:16:02.192479 7324 external_mini_cluster.cc:1428] Setting key 4829639e3cf1cef1a48e110690bf0da4
Aug 11 12:16:02 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:02.211701 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:02.190959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:33347 (local address 127.7.39.62:44691)
0811 12:16:02.191484 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:02.191490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:02.192637 (+ 1147us) server_negotiation.cc:407] Connection header received
0811 12:16:02.194787 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:02.194791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:02.194855 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:02.194946 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:02.197087 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:02.197921 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:02.200095 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:02.200503 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:02.200602 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:02.204965 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:02.204973 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:02.204977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:02.205025 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:02.208312 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:02.209125 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:02.209131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:02.209134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:02.209329 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:02.209793 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:02.209798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:02.209800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:02.210145 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:02.210293 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:02.211113 (+ 820us) server_negotiation.cc:299] Negotiation successful
0811 12:16:02.211358 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":274,"threads_started":1}
I20260811 12:16:02.212658 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:02.191466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44691 (local address 127.7.39.1:33347)
0811 12:16:02.192388 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:02.192453 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:02.194490 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:02.195256 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:02.195269 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:02.196021 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:02.196895 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:02.196924 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:02.198105 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:02.198116 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:02.199957 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:02.199974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:02.200754 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:02.200761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:02.201059 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:02.201861 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:02.201902 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:02.204674 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:02.208480 (+ 3806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:02.208488 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:02.208503 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:02.208989 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:02.209452 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:02.209457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:02.209461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:02.209678 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:02.210291 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:02.210300 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:02.210726 (+ 426us) client_negotiation.cc:769] Sending connection context
0811 12:16:02.210977 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 12:16:02.211430 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":348,"threads_started":1}
I20260811 12:16:02.214370 16584 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44691
I20260811 12:16:02.214666 16584 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:02.215546 16584 heartbeater.cc:511] Master 127.7.39.62:44691 requested a full tablet report, sending...
I20260811 12:16:02.217844 16281 ts_manager.cc:194] Registered new tserver with Master: 39a84466928d4d77b7d698c014a4ea21 (127.7.39.1:35487)
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
I20260811 12:16:02.219357 16281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:33347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:16:02.252430 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44691
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:02.592837 16596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:02.593207 16596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:02.593361 16596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:02.600678 16596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:02.600816 16596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:02.600962 16596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:02.611025 16596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:02.612394 16596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:02.613935 16596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:02.627064 16602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:02.627040 16601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:02.627102 16605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:02.628171 16596 server_base.cc:1034] running on GCE node
I20260811 12:16:02.628765 16596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:02.629760 16596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:02.630967 16596 hybrid_clock.cc:648] HybridClock initialized: now 1786450562630913 us; error 45 us; skew 500 ppm
I20260811 12:16:02.631287 16596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:02 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:02.634580 16596 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:16:02.636104 16596 webserver.cc:460] Webserver started at http://127.7.39.2:41601/ using document root <none> and password file <none>
I20260811 12:16:02.636812 16596 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:02.636924 16596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:02.637259 16596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:02.640481 16596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "11a061cb871d423abe2120724e108ae5"
format_stamp: "Formatted at 2026-08-11 12:16:02 on dist-test-slave-rmhg"
server_key: "1e6a96b24cfc5d860a7a384e9247abc9"
server_key_iv: "059b4b5674fd84249f43ace99f906c9f"
server_key_version: "encryptionkey@0"
I20260811 12:16:02.641335 16596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "11a061cb871d423abe2120724e108ae5"
format_stamp: "Formatted at 2026-08-11 12:16:02 on dist-test-slave-rmhg"
server_key: "1e6a96b24cfc5d860a7a384e9247abc9"
server_key_iv: "059b4b5674fd84249f43ace99f906c9f"
server_key_version: "encryptionkey@0"
I20260811 12:16:02.646934 16596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:16:02.650597 16617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:02.652019 16596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:02.652169 16596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "11a061cb871d423abe2120724e108ae5"
format_stamp: "Formatted at 2026-08-11 12:16:02 on dist-test-slave-rmhg"
server_key: "1e6a96b24cfc5d860a7a384e9247abc9"
server_key_iv: "059b4b5674fd84249f43ace99f906c9f"
server_key_version: "encryptionkey@0"
I20260811 12:16:02.652350 16596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:02.679695 16596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:02.684109 16596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:02.684399 16596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:02.685483 16596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:02.686906 16596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:02.686986 16596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:02.687080 16596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:02.687124 16596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:02.718585 16596 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:42607
I20260811 12:16:02.718667 16803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:42607 every 8 connection(s)
I20260811 12:16:02.719859 16596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:02.727782 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 16596
I20260811 12:16:02.727973 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:16:02.728487 7324 external_mini_cluster.cc:1428] Setting key 3440bc9866d677ac20501264b86d81e3
Aug 11 12:16:02 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450562, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:02.742494 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:02.722523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:48375 (local address 127.7.39.62:44691)
0811 12:16:02.722988 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:02.722994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:02.724132 (+ 1138us) server_negotiation.cc:407] Connection header received
0811 12:16:02.726251 (+ 2119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:02.726256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:02.726331 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:02.726434 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:02.728237 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:02.728982 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:02.731073 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:02.731514 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:02.731617 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:02.736011 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:02.736017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:02.736022 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:02.736068 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:02.739156 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:02.739985 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:02.739991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:02.739994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:02.740160 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:02.740564 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:02.740569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:02.740571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:02.740910 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:02.741027 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:02.742011 (+ 984us) server_negotiation.cc:299] Negotiation successful
0811 12:16:02.742207 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":211,"threads_started":1}
I20260811 12:16:02.743961 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:02.723123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44691 (local address 127.7.39.2:48375)
0811 12:16:02.723943 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:02.723995 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:02.725965 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:02.726666 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:02.726675 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:02.727215 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:02.728047 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:02.728076 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:02.729206 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:02.729210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:02.730900 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:02.730918 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:02.731771 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:02.731778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:02.732087 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:02.733013 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:02.733045 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:02.735714 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:02.739335 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:02.739344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:02.739361 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:02.739839 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:02.740254 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:02.740258 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:02.740260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:02.740456 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:02.741035 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:02.741044 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:02.741574 (+ 530us) client_negotiation.cc:769] Sending connection context
0811 12:16:02.741908 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 12:16:02.742523 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":317,"threads_started":1}
I20260811 12:16:02.745647 16804 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44691
I20260811 12:16:02.746004 16804 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:02.746986 16804 heartbeater.cc:511] Master 127.7.39.62:44691 requested a full tablet report, sending...
I20260811 12:16:02.748721 16281 ts_manager.cc:194] Registered new tserver with Master: 11a061cb871d423abe2120724e108ae5 (127.7.39.2:42607)
I20260811 12:16:02.749593 16281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:48375
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:16:02.785844 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44691
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:03.120743 16813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:03.121064 16813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:03.121244 16813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:03.128985 16813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:03.129120 16813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:03.129323 16813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:03.139496 16813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:03.140874 16813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:03.142325 16813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:03.155452 16821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:03.155484 16825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:03.155506 16823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:03.156335 16813 server_base.cc:1034] running on GCE node
I20260811 12:16:03.156980 16813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:03.158010 16813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:03.159219 16813 hybrid_clock.cc:648] HybridClock initialized: now 1786450563159156 us; error 54 us; skew 500 ppm
I20260811 12:16:03.159538 16813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:03.162732 16813 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:16:03.164129 16813 webserver.cc:460] Webserver started at http://127.7.39.3:37481/ using document root <none> and password file <none>
I20260811 12:16:03.164837 16813 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:03.164959 16813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:03.165309 16813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:03.168313 16813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "c22d3b121ae84e3eb7aafbf86228866c"
format_stamp: "Formatted at 2026-08-11 12:16:03 on dist-test-slave-rmhg"
server_key: "cda3b7710f246afeeddb95eee1a6736c"
server_key_iv: "b6f9c778161c98112ac4e29c3c7c7bb0"
server_key_version: "encryptionkey@0"
I20260811 12:16:03.169250 16813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "c22d3b121ae84e3eb7aafbf86228866c"
format_stamp: "Formatted at 2026-08-11 12:16:03 on dist-test-slave-rmhg"
server_key: "cda3b7710f246afeeddb95eee1a6736c"
server_key_iv: "b6f9c778161c98112ac4e29c3c7c7bb0"
server_key_version: "encryptionkey@0"
I20260811 12:16:03.174804 16813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:16:03.178328 16839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:03.179715 16813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:03.179877 16813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "c22d3b121ae84e3eb7aafbf86228866c"
format_stamp: "Formatted at 2026-08-11 12:16:03 on dist-test-slave-rmhg"
server_key: "cda3b7710f246afeeddb95eee1a6736c"
server_key_iv: "b6f9c778161c98112ac4e29c3c7c7bb0"
server_key_version: "encryptionkey@0"
I20260811 12:16:03.180069 16813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:03.195549 16813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:03.200029 16813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:03.200312 16813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:03.201326 16813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:03.202811 16813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:03.202891 16813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:03.203014 16813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:03.203073 16813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:03.222534 16584 heartbeater.cc:503] Master 127.7.39.62:44691 was elected leader, sending a full tablet report...
I20260811 12:16:03.234565 16813 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:38325
I20260811 12:16:03.234615 17016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:38325 every 8 connection(s)
I20260811 12:16:03.235852 16813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:03.240568 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 16813
I20260811 12:16:03.240734 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:16:03.241220 7324 external_mini_cluster.cc:1428] Setting key e7899d5b250e40d4c7f1bfc4cb8c5946
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:03.255959 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:03.238554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:51113 (local address 127.7.39.62:44691)
0811 12:16:03.238741 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.238746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.239989 (+ 1243us) server_negotiation.cc:407] Connection header received
0811 12:16:03.241984 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.241989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.242085 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.242172 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.243915 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.244819 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.246483 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.246872 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.246965 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:03.250356 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:03.250360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:03.250364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:03.250398 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:03.252921 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.253623 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.253626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.253629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.253752 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.254111 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.254114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.254116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.254393 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:03.254497 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:03.255366 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 12:16:03.255600 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:16:03.257215 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:03.239020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44691 (local address 127.7.39.3:51113)
0811 12:16:03.239817 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:03.239858 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:03.241688 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:03.242418 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:03.242431 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:03.243023 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:03.243738 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:03.243758 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.244972 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.244976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:03.246340 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:03.246354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.247058 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.247063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:03.247286 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:03.247920 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:03.247943 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:03.250155 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:03.253039 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:03.253046 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:03.253059 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:03.253522 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:03.253833 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:03.253836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:03.253839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:03.254031 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:03.254483 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:03.254489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:03.254955 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 12:16:03.255229 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 12:16:03.255777 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":304,"threads_started":1}
I20260811 12:16:03.258630 17017 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44691
I20260811 12:16:03.258957 17017 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:03.259737 17017 heartbeater.cc:511] Master 127.7.39.62:44691 requested a full tablet report, sending...
I20260811 12:16:03.261183 16281 ts_manager.cc:194] Registered new tserver with Master: c22d3b121ae84e3eb7aafbf86228866c (127.7.39.3:38325)
I20260811 12:16:03.262005 16281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:51113
I20260811 12:16:03.269342 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/test-admin.keytab.
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:03.300947 7324 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 12:16:03.306555 17027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:03.303922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.7.39.1:35487)
0811 12:16:03.304602 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.304610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.304662 (+ 52us) server_negotiation.cc:407] Connection header received
0811 12:16:03.304843 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.304850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.305084 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.305397 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.306189 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52382: BlockingRecv error: recv got EOF from 127.0.0.1:52382 (error 108)
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":368,"threads_started":1}
W20260811 12:16:03.306718 7324 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:03.308353 7324 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:03.309705 7324 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:16:03.325948 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:03.311561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.7.39.1:35487)
0811 12:16:03.311765 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.311771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.312028 (+ 257us) server_negotiation.cc:407] Connection header received
0811 12:16:03.312239 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.312243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.312313 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.312399 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.313528 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.314622 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.315963 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.316601 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.316725 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:03.320430 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:03.320445 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:03.320464 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:03.320505 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:03.323222 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.323912 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.323918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.323921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.324045 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.324328 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.324331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.324333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.324735 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:03.324963 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:03.325298 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 12:16:03.325552 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 12:16:03.332078 17027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:03.330151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.7.39.1:35487)
0811 12:16:03.330328 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.330334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.330621 (+ 287us) server_negotiation.cc:407] Connection header received
0811 12:16:03.330805 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.330809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.330871 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.330951 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.331805 (+ 854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52410: BlockingRecv error: recv got EOF from 127.0.0.1:52410 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 12:16:03.332285 7324 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:03.334033 7324 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:03.335393 7324 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:16:03.349805 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:03.336877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.7.39.1:35487)
0811 12:16:03.337100 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.337106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.337323 (+ 217us) server_negotiation.cc:407] Connection header received
0811 12:16:03.337466 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.337470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.337535 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.337617 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.338615 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.339342 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.340553 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.341014 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.341116 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:03.344787 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:03.344793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:03.344796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:03.344836 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:03.347441 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.348108 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.348111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.348114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.348254 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.348547 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.348550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.348552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.348850 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:03.349050 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:03.349327 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 12:16:03.349500 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 12:16:03.354296 17027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:03.352473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.7.39.1:35487)
0811 12:16:03.352645 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.352650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.353030 (+ 380us) server_negotiation.cc:407] Connection header received
0811 12:16:03.353202 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.353206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.353283 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.353355 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.354037 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52418: BlockingRecv error: recv got EOF from 127.0.0.1:52418 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 12:16:03.354444 7324 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450508543501-7324-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:03.356014 7324 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:03.357375 7324 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:16:03.371865 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:03.358793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52422 (local address 127.7.39.1:35487)
0811 12:16:03.358991 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.358997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.359141 (+ 144us) server_negotiation.cc:407] Connection header received
0811 12:16:03.359303 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.359306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.359365 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.359443 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.360423 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.361193 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.362467 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.362896 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.363011 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:03.366689 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:03.366694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:03.366698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:03.366735 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:03.369123 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.369850 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.369856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.369859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.370029 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.370398 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.370402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.370405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.370727 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:03.371035 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:03.371323 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 12:16:03.371541 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:16:03.373582 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 16382
I20260811 12:16:03.385335 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:03.374300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.7.39.1:35487)
0811 12:16:03.374484 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.374490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.374821 (+ 331us) server_negotiation.cc:407] Connection header received
0811 12:16:03.374935 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.374939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.375000 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.375103 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.376080 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.376775 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.378010 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.378450 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.378570 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:03.380493 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:03.380498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:03.380502 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:03.380532 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:03.382743 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.383360 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.383365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.383367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.383498 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.383850 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.383855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.383858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.384168 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:03.384417 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:03.384741 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:16:03.384959 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:16:03.385980 16579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:03.469444 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 16382
I20260811 12:16:03.491086 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 16596
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:16:03.507237 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:03.491998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33640 (local address 127.7.39.2:42607)
0811 12:16:03.492581 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.492589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.492618 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:16:03.492682 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.492689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.492898 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.493071 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.494110 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.495185 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.496468 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.497092 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.497387 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:03.501303 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:03.501316 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:03.501334 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:03.501373 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:03.504283 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.505024 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.505031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.505033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.505206 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.505595 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.505600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.505604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.505972 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:03.506267 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:03.506641 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 12:16:03.506903 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":308,"threads_started":1}
I20260811 12:16:03.508594 16795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:03.599346 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 16596
I20260811 12:16:03.621239 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 16813
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[16208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:16:03.637588 17052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:03.622419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47462 (local address 127.7.39.3:38325)
0811 12:16:03.623167 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:03.623174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:03.623208 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:16:03.623271 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:03.623278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:03.623500 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:03.623657 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:03.624696 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.625799 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.627013 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:03.627580 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:03.627697 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:03.631800 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:03.631815 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:03.631833 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:03.631874 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:03.634846 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.635596 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.635603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.635605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.635752 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:03.636057 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:03.636060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:03.636062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:03.636437 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:03.636719 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:03.636958 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 12:16:03.637260 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":410,"threads_started":1}
I20260811 12:16:03.639112 17006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:03.723871 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 16813
I20260811 12:16:03.748119 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 16227
I20260811 12:16:03.749421 16330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:03.854553 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 16227
2026-08-11T12:16:03Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4258 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[17064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[17064](info): set up 2 sockets
Aug 11 12:16:03 dist-test-slave-rmhg krb5kdc[17064](info): commencing operation
krb5kdc: starting...
W20260811 12:16:05.116602 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:16:05 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:16:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.62.keytab.
Entry for principal oryx/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.62.keytab.
I20260811 12:16:05.382961 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:37677
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.62.keytab
--principal=oryx/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:37677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:05.730896 17088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:05.731221 17088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:05.731379 17088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:05.739032 17088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:05.739169 17088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:05.739243 17088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:05.739322 17088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:05.739380 17088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:05.749908 17088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:37677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.62.keytab
--principal=oryx/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:37677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:05.751266 17088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:05.752621 17088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:05.765108 17097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:05.765144 17098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:05.765368 17101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:05.766000 17088 server_base.cc:1034] running on GCE node
I20260811 12:16:05.766780 17088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:05.768249 17088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:05.769472 17088 hybrid_clock.cc:648] HybridClock initialized: now 1786450565769413 us; error 45 us; skew 500 ppm
I20260811 12:16:05.769982 17088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:05 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:05.773092 17088 init.cc:377] Logged in from keytab as oryx/127.7.39.62@KRBTEST.COM (short username oryx)
I20260811 12:16:05.774710 17088 webserver.cc:460] Webserver started at http://127.7.39.62:34481/ using document root <none> and password file <none>
I20260811 12:16:05.775416 17088 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:05.775542 17088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:05.775844 17088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:05.779018 17088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "136c6fa878384eb88036b9fa7c3377b2"
format_stamp: "Formatted at 2026-08-11 12:16:05 on dist-test-slave-rmhg"
server_key: "336fb5c9fc6ca6ff480df114996a7df2"
server_key_iv: "7976e48f855bbf1bb1f5f2d9b92504ec"
server_key_version: "encryptionkey@0"
I20260811 12:16:05.779817 17088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "136c6fa878384eb88036b9fa7c3377b2"
format_stamp: "Formatted at 2026-08-11 12:16:05 on dist-test-slave-rmhg"
server_key: "336fb5c9fc6ca6ff480df114996a7df2"
server_key_iv: "7976e48f855bbf1bb1f5f2d9b92504ec"
server_key_version: "encryptionkey@0"
I20260811 12:16:05.785444 17088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:16:05.788980 17113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:05.790489 17088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:05.790690 17088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "136c6fa878384eb88036b9fa7c3377b2"
format_stamp: "Formatted at 2026-08-11 12:16:05 on dist-test-slave-rmhg"
server_key: "336fb5c9fc6ca6ff480df114996a7df2"
server_key_iv: "7976e48f855bbf1bb1f5f2d9b92504ec"
server_key_version: "encryptionkey@0"
I20260811 12:16:05.790884 17088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:05.810380 17088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:05.814888 17088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:05.815194 17088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:05.833603 17088 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:37677
I20260811 12:16:05.833653 17195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:37677 every 8 connection(s)
I20260811 12:16:05.835039 17088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:16:05.838760 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 17088
I20260811 12:16:05.838959 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:16:05.839020 17196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:05.839605 7324 external_mini_cluster.cc:1428] Setting key 19459fe3d6468cd56227db3eb34057d8
I20260811 12:16:05.853849 17196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: Bootstrap starting.
Aug 11 12:16:05 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:05.858115 17196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:05.859570 17196 log.cc:824] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:05.863336 17196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: No bootstrap required, opened a new log
I20260811 12:16:05.864374 17200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:05.841690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.7.39.62:37677)
0811 12:16:05.842547 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:05.842562 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:05.842615 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:16:05.844223 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:05.844254 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:05.844812 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:05.845320 (+ 508us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:05.846629 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:05.848598 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:05.849974 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:05.851103 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:05.851324 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:05.855402 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:05.855416 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:05.855438 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:05.855497 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:05.859397 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:05.860228 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:05.860237 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:05.860250 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:05.860437 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:05.860885 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:05.860889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:05.860891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:05.861715 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:05.862022 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:05.862426 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 12:16:05.862900 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":368,"threads_started":1}
I20260811 12:16:05.871567 17196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } }
I20260811 12:16:05.871982 17196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:05.872062 17196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 136c6fa878384eb88036b9fa7c3377b2, State: Initialized, Role: FOLLOWER
I20260811 12:16:05.872581 17196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } }
I20260811 12:16:05.872814 17196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:16:05.872964 17196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:16:05.873176 17196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:05.875540 17196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } }
I20260811 12:16:05.876051 17196 leader_election.cc:302] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 136c6fa878384eb88036b9fa7c3377b2; no voters:
I20260811 12:16:05.876631 17196 leader_election.cc:288] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:16:05.876771 17203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:05.877432 17203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 1 LEADER]: Becoming Leader. State: Replica: 136c6fa878384eb88036b9fa7c3377b2, State: Running, Role: LEADER
I20260811 12:16:05.877995 17203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } }
I20260811 12:16:05.878707 17196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:05.880607 17204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } }
I20260811 12:16:05.881052 17204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:05.881985 17219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:05.881809 17211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 136c6fa878384eb88036b9fa7c3377b2. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } }
I20260811 12:16:05.882266 17211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:05.887763 17219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:05.897605 17219 catalog_manager.cc:1347] Generated new cluster ID: 7a252f3444ae484aa05fd7c3110b1bfe
I20260811 12:16:05.897730 17219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:05.912093 17219 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:16:05.913519 17219 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:05.925673 17219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: Generated new TSK 0
I20260811 12:16:05.926609 17219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab.
Entry for principal oryx/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab.
I20260811 12:16:05.997049 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37677
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab
--principal=oryx/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:06.341279 17250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:06.341674 17250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:06.341835 17250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:06.349340 17250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:06.349486 17250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:06.349642 17250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:06.359926 17250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab
--principal=oryx/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:06.361516 17250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:06.363025 17250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:06.376919 17265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:06.376858 17262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:06.377714 17261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:06.378121 17250 server_base.cc:1034] running on GCE node
I20260811 12:16:06.378981 17250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:06.380366 17250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:06.381589 17250 hybrid_clock.cc:648] HybridClock initialized: now 1786450566381510 us; error 67 us; skew 500 ppm
I20260811 12:16:06.382010 17250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:06 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450566, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:06.385908 17250 init.cc:377] Logged in from keytab as oryx/127.7.39.1@KRBTEST.COM (short username oryx)
I20260811 12:16:06.387450 17250 webserver.cc:460] Webserver started at http://127.7.39.1:41325/ using document root <none> and password file <none>
I20260811 12:16:06.388170 17250 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:06.388288 17250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:06.388597 17250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:06.391784 17250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "9821077cf82e4c0c941244fbee6abf1e"
format_stamp: "Formatted at 2026-08-11 12:16:06 on dist-test-slave-rmhg"
server_key: "6f37fa804df23b53e13eacff89994f63"
server_key_iv: "c9262b205eef637cddaf6b03671fc346"
server_key_version: "encryptionkey@0"
I20260811 12:16:06.392578 17250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "9821077cf82e4c0c941244fbee6abf1e"
format_stamp: "Formatted at 2026-08-11 12:16:06 on dist-test-slave-rmhg"
server_key: "6f37fa804df23b53e13eacff89994f63"
server_key_iv: "c9262b205eef637cddaf6b03671fc346"
server_key_version: "encryptionkey@0"
I20260811 12:16:06.398445 17250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 12:16:06.402483 17279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:06.404067 17250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:16:06.404243 17250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "9821077cf82e4c0c941244fbee6abf1e"
format_stamp: "Formatted at 2026-08-11 12:16:06 on dist-test-slave-rmhg"
server_key: "6f37fa804df23b53e13eacff89994f63"
server_key_iv: "c9262b205eef637cddaf6b03671fc346"
server_key_version: "encryptionkey@0"
I20260811 12:16:06.404480 17250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:06.419379 17250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:06.423735 17250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:06.424031 17250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:06.425060 17250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:06.426600 17250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:06.426676 17250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:06.426772 17250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:06.426819 17250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:06.458279 17250 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:35645
I20260811 12:16:06.458328 17471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:35645 every 8 connection(s)
I20260811 12:16:06.459560 17250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:06.464006 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 17250
I20260811 12:16:06.464210 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:16:06.464875 7324 external_mini_cluster.cc:1428] Setting key 451dd0aa67d81179cb1486d5a3b36549
Aug 11 12:16:06 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450566, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:06.480834 17478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:06.462177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:51895 (local address 127.7.39.62:37677)
0811 12:16:06.462601 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:06.462605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:06.463508 (+ 903us) server_negotiation.cc:407] Connection header received
0811 12:16:06.465515 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:06.465521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:06.465606 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:06.465727 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:06.467663 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:06.468637 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:06.470343 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:06.470880 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:06.471002 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:06.474626 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:06.474633 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:06.474638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:06.474679 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:06.477542 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:06.478482 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:06.478488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:06.478492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:06.478615 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:06.479073 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:06.479076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:06.479078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:06.479393 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:06.479520 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:06.480384 (+ 864us) server_negotiation.cc:299] Negotiation successful
0811 12:16:06.480571 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":219,"threads_started":1}
I20260811 12:16:06.482337 17479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:06.462627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:51895)
0811 12:16:06.463342 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:06.463382 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:06.465226 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:06.465995 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:06.466007 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:06.466786 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:06.467473 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:06.467492 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:06.468826 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:06.468830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:06.470181 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:06.470194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:06.471090 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:06.471095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:06.471305 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:06.471962 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:06.471986 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:06.474368 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:06.477706 (+ 3338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:06.477716 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:06.477735 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:06.478365 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:06.478716 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:06.478721 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:06.478724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:06.478976 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:06.479520 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:06.479529 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:06.479974 (+ 445us) client_negotiation.cc:769] Sending connection context
0811 12:16:06.480284 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 12:16:06.480893 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":269,"threads_started":1}
I20260811 12:16:06.484068 17473 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37677
I20260811 12:16:06.484437 17473 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:06.485441 17473 heartbeater.cc:511] Master 127.7.39.62:37677 requested a full tablet report, sending...
I20260811 12:16:06.487987 17134 ts_manager.cc:194] Registered new tserver with Master: 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645)
I20260811 12:16:06.490083 17134 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:51895
WARNING: no policy specified for oryx/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab.
Entry for principal oryx/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab.
I20260811 12:16:06.526131 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37677
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab
--principal=oryx/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:06.868882 17488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:06.869230 17488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:06.869375 17488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:06.876686 17488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:06.876828 17488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:06.877022 17488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:06.887205 17488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab
--principal=oryx/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:06.888551 17488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:06.889977 17488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:06.903631 17501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:06.903972 17496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:06.904327 17488 server_base.cc:1034] running on GCE node
W20260811 12:16:06.904386 17497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:06.905079 17488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:06.906071 17488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:06.907271 17488 hybrid_clock.cc:648] HybridClock initialized: now 1786450566907217 us; error 48 us; skew 500 ppm
I20260811 12:16:06.907616 17488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:06 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450566, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:06.910948 17488 init.cc:377] Logged in from keytab as oryx/127.7.39.2@KRBTEST.COM (short username oryx)
I20260811 12:16:06.912451 17488 webserver.cc:460] Webserver started at http://127.7.39.2:41629/ using document root <none> and password file <none>
I20260811 12:16:06.913223 17488 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:06.913359 17488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:06.913672 17488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:06.916680 17488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "0c4039332f2847d9b9743590f94bafe1"
format_stamp: "Formatted at 2026-08-11 12:16:06 on dist-test-slave-rmhg"
server_key: "a79a596747cf1e95c76a25ba62931b53"
server_key_iv: "9bfc31c46ea32f549660534413f757f1"
server_key_version: "encryptionkey@0"
I20260811 12:16:06.917492 17488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "0c4039332f2847d9b9743590f94bafe1"
format_stamp: "Formatted at 2026-08-11 12:16:06 on dist-test-slave-rmhg"
server_key: "a79a596747cf1e95c76a25ba62931b53"
server_key_iv: "9bfc31c46ea32f549660534413f757f1"
server_key_version: "encryptionkey@0"
I20260811 12:16:06.922751 17488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:16:06.926343 17512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:06.927714 17488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:16:06.927853 17488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "0c4039332f2847d9b9743590f94bafe1"
format_stamp: "Formatted at 2026-08-11 12:16:06 on dist-test-slave-rmhg"
server_key: "a79a596747cf1e95c76a25ba62931b53"
server_key_iv: "9bfc31c46ea32f549660534413f757f1"
server_key_version: "encryptionkey@0"
I20260811 12:16:06.928020 17488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:06.966848 17488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:06.970958 17488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:06.971226 17488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:06.972203 17488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:06.973637 17488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:06.973714 17488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:06.973806 17488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:06.973853 17488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:07.005412 17488 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:37651
I20260811 12:16:07.005491 17682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:37651 every 8 connection(s)
I20260811 12:16:07.006711 17488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:07.012409 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 17488
I20260811 12:16:07.012576 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:16:07.013170 7324 external_mini_cluster.cc:1428] Setting key 8db0734d6de534bfed400f9048b93179
Aug 11 12:16:07 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450566, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:07.027808 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:07.009426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:58647 (local address 127.7.39.62:37677)
0811 12:16:07.009884 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:07.009888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:07.010762 (+ 874us) server_negotiation.cc:407] Connection header received
0811 12:16:07.012660 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:07.012666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:07.012759 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:07.012879 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:07.014645 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:07.015582 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:07.017210 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:07.017814 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:07.017937 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:07.021349 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:07.021356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:07.021360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:07.021402 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:07.024464 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:07.025285 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:07.025289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:07.025292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:07.025425 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:07.025968 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:07.025972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:07.025974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:07.026263 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:07.026378 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:07.027305 (+ 927us) server_negotiation.cc:299] Negotiation successful
0811 12:16:07.027488 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":238,"threads_started":1}
I20260811 12:16:07.029268 17687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:07.009883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:58647)
0811 12:16:07.010594 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:07.010635 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:07.012342 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:07.013068 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:07.013077 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:07.013682 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:07.014462 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:07.014489 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:07.015738 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:07.015742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:07.017033 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:07.017046 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:07.018009 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:07.018013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:07.018231 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:07.018860 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:07.018884 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:07.021088 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:07.024641 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:07.024649 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:07.024673 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:07.025133 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:07.025526 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:07.025531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:07.025534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:07.025836 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:07.026403 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:07.026412 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:07.026889 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 12:16:07.027196 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 12:16:07.027781 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":265,"threads_started":1}
I20260811 12:16:07.030933 17683 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37677
I20260811 12:16:07.031279 17683 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:07.032100 17683 heartbeater.cc:511] Master 127.7.39.62:37677 requested a full tablet report, sending...
I20260811 12:16:07.033766 17134 ts_manager.cc:194] Registered new tserver with Master: 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:07.034904 17134 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:58647
WARNING: no policy specified for oryx/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab.
Entry for principal oryx/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab.
I20260811 12:16:07.070642 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37677
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab
--principal=oryx/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:07.416671 17691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:07.416954 17691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:07.417048 17691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:07.424472 17691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:07.424592 17691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:07.424726 17691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:07.434779 17691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab
--principal=oryx/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:37677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:07.436084 17691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:07.437453 17691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:07.450531 17698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:07.450495 17699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:07.450552 17702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:07.451860 17691 server_base.cc:1034] running on GCE node
I20260811 12:16:07.452502 17691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:07.453568 17691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:07.454778 17691 hybrid_clock.cc:648] HybridClock initialized: now 1786450567454716 us; error 58 us; skew 500 ppm
I20260811 12:16:07.455078 17691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:07 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450567, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:07.458729 17691 init.cc:377] Logged in from keytab as oryx/127.7.39.3@KRBTEST.COM (short username oryx)
I20260811 12:16:07.460598 17691 webserver.cc:460] Webserver started at http://127.7.39.3:45443/ using document root <none> and password file <none>
I20260811 12:16:07.461747 17691 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:07.461902 17691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:07.462297 17691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:07.466017 17691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "9a2cc2bb2de3418daf003e6958c41172"
format_stamp: "Formatted at 2026-08-11 12:16:07 on dist-test-slave-rmhg"
server_key: "5b107993ad306ac89c33d27128799513"
server_key_iv: "ae626073cd104594debed7eaa900a0fb"
server_key_version: "encryptionkey@0"
I20260811 12:16:07.466764 17691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "9a2cc2bb2de3418daf003e6958c41172"
format_stamp: "Formatted at 2026-08-11 12:16:07 on dist-test-slave-rmhg"
server_key: "5b107993ad306ac89c33d27128799513"
server_key_iv: "ae626073cd104594debed7eaa900a0fb"
server_key_version: "encryptionkey@0"
I20260811 12:16:07.472106 17691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:16:07.475653 17711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:07.477074 17691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:16:07.477260 17691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "9a2cc2bb2de3418daf003e6958c41172"
format_stamp: "Formatted at 2026-08-11 12:16:07 on dist-test-slave-rmhg"
server_key: "5b107993ad306ac89c33d27128799513"
server_key_iv: "ae626073cd104594debed7eaa900a0fb"
server_key_version: "encryptionkey@0"
I20260811 12:16:07.477450 17691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:07.494021 17473 heartbeater.cc:503] Master 127.7.39.62:37677 was elected leader, sending a full tablet report...
I20260811 12:16:07.527451 17691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:07.532078 17691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:07.532336 17691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:07.533366 17691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:07.534722 17691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:07.534798 17691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:07.534891 17691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:07.534935 17691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:07.565827 17691 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:42173
I20260811 12:16:07.565874 17904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:42173 every 8 connection(s)
I20260811 12:16:07.567181 17691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:07.569569 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 17691
I20260811 12:16:07.569777 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:16:07.570291 7324 external_mini_cluster.cc:1428] Setting key 713a53b9871a40e2b619f85b0253bf39
Aug 11 12:16:07 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450567, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:07.588774 17910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:07.569862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:53767 (local address 127.7.39.62:37677)
0811 12:16:07.570441 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:07.570448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:07.571579 (+ 1131us) server_negotiation.cc:407] Connection header received
0811 12:16:07.574011 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:07.574016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:07.574098 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:07.574197 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:07.575729 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:07.576513 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:07.578191 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:07.578638 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:07.578751 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:07.582461 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:07.582466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:07.582469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:07.582507 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:07.585584 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:07.586593 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:07.586598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:07.586601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:07.586772 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:07.587170 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:07.587173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:07.587175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:07.587447 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:07.587561 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:07.588311 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 12:16:07.588490 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":267,"threads_started":1}
I20260811 12:16:07.590057 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:07.570520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:53767)
0811 12:16:07.571407 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:07.571459 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:07.573744 (+ 2285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:07.574410 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:07.574419 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:07.574955 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:07.575587 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:07.575606 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:07.576655 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:07.576659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:07.578051 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:07.578065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:07.578852 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:07.578857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:07.579095 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:07.579834 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:07.579870 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:07.582233 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:07.585768 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:07.585780 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:07.585802 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:07.586456 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:07.586867 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:07.586871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:07.586873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:07.587078 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:07.587542 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:07.587549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:07.587928 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 12:16:07.588176 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 12:16:07.588623 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":333,"threads_started":1}
I20260811 12:16:07.591471 17906 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37677
I20260811 12:16:07.591789 17906 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:07.592516 17906 heartbeater.cc:511] Master 127.7.39.62:37677 requested a full tablet report, sending...
I20260811 12:16:07.594028 17134 ts_manager.cc:194] Registered new tserver with Master: 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173)
I20260811 12:16:07.594913 17134 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:53767
I20260811 12:16:07.598968 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.39.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.61.keytab.
Entry for principal oryx/127.7.39.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.61.keytab.
I20260811 12:16:07.656596 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.61:39781
--webserver_interface=127.7.39.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.61.keytab
--principal=oryx/127.7.39.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:37677,127.7.39.61:39781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:08.002902 17914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:08.003239 17914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:08.003377 17914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:08.011502 17914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:08.011632 17914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:08.011713 17914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:08.011788 17914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:08.011847 17914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:08.022140 17914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:37677,127.7.39.61:39781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.61.keytab
--principal=oryx/127.7.39.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.61:39781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.39.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
W20260811 12:16:08.022541 17914 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.39.62:37677,127.7.39.61:39781'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 12:16:08.023682 17914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:08.025178 17914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:08.037245 17920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:08.037209 17923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:08.037209 17925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:08.037763 17683 heartbeater.cc:503] Master 127.7.39.62:37677 was elected leader, sending a full tablet report...
I20260811 12:16:08.038968 17914 server_base.cc:1034] running on GCE node
I20260811 12:16:08.039871 17914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:08.041385 17914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:08.042615 17914 hybrid_clock.cc:648] HybridClock initialized: now 1786450568042540 us; error 46 us; skew 500 ppm
I20260811 12:16:08.042961 17914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450568, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:08.046139 17914 init.cc:377] Logged in from keytab as oryx/127.7.39.61@KRBTEST.COM (short username oryx)
I20260811 12:16:08.047609 17914 webserver.cc:460] Webserver started at http://127.7.39.61:34251/ using document root <none> and password file <none>
I20260811 12:16:08.048318 17914 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:08.048425 17914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:08.048753 17914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:08.051921 17914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data/instance:
uuid: "1b5602e8998c40879c210fb36631f753"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "fc0edcdd1d825fdb7382e4457f0217ab"
server_key_iv: "a434fb747dc2807e468ebc17b8992cb5"
server_key_version: "encryptionkey@0"
I20260811 12:16:08.052726 17914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal/instance:
uuid: "1b5602e8998c40879c210fb36631f753"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "fc0edcdd1d825fdb7382e4457f0217ab"
server_key_iv: "a434fb747dc2807e468ebc17b8992cb5"
server_key_version: "encryptionkey@0"
I20260811 12:16:08.058192 17914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 12:16:08.061796 17940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:08.063191 17914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:08.063362 17914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
uuid: "1b5602e8998c40879c210fb36631f753"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "fc0edcdd1d825fdb7382e4457f0217ab"
server_key_iv: "a434fb747dc2807e468ebc17b8992cb5"
server_key_version: "encryptionkey@0"
I20260811 12:16:08.063570 17914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:08.083374 17914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:08.087833 17914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:08.088081 17914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:08.106542 17914 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.61:39781
I20260811 12:16:08.106598 18014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.61:39781 every 8 connection(s)
I20260811 12:16:08.108057 17914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
I20260811 12:16:08.112095 18015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:08.112689 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 17914
I20260811 12:16:08.121845 18015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.61@KRBTEST.COM
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450568, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.61@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:08.138298 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.115156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.7.39.61:39781)
0811 12:16:08.116002 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.116015 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.116060 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:16:08.118525 (+ 2465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.118560 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.119043 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.119430 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.121432 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.122951 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.124740 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.125797 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.126005 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.130163 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.130171 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.130183 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.130223 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.133673 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.134409 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.134416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.134423 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.134601 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.135096 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.135102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.135105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.135759 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.135991 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.136513 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.136909 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":398,"threads_started":1}
I20260811 12:16:08.140404 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.125085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.7.39.62:37677)
0811 12:16:08.125673 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.125679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.126259 (+ 580us) server_negotiation.cc:407] Connection header received
0811 12:16:08.126454 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.126459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.126558 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.126656 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.127806 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.128582 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.130078 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.130506 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.130602 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.134369 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.134376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.134381 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.134426 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.137461 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.138318 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.138321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.138323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.138474 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.138878 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.138882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.138884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.139219 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.139356 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.139843 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.140063 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":312,"threads_started":1}
I20260811 12:16:08.140482 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.125085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:48760)
0811 12:16:08.126079 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.126135 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.126318 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.126755 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.126762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.127389 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.127649 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.127658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.128724 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.128728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.129894 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.129908 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.130698 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.130705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.130865 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.131466 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.131488 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.134156 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.137588 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.137599 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.137606 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.138201 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.138590 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.138596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.138600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.138739 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.139353 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.139361 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.139522 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.139711 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.140054 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":577,"threads_started":1}
I20260811 12:16:08.142406 18015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450568, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.61@KRBTEST.COM for oryx/127.7.39.61@KRBTEST.COM
I20260811 12:16:08.154633 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.143182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.0.0.1:35498)
0811 12:16:08.143363 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.143380 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.143527 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.143798 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.143802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.144161 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.144289 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.144297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.145247 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.145252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.146146 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.146158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.146803 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.146808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.146998 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.147553 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.147570 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.149611 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.152326 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.152330 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.152333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.152731 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.153114 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.153118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.153120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.153312 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.153858 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.153861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.153991 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.154114 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.154348 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:16:08.154820 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.143158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35498 (local address 127.7.39.61:39781)
0811 12:16:08.143357 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.143363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.143441 (+ 78us) server_negotiation.cc:407] Connection header received
0811 12:16:08.143630 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.143635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.143691 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.143767 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.144419 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.145115 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.146273 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.146664 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.146772 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.149805 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.149810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.149813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.149842 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.152186 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.152871 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.152876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.152878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.153019 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.153407 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.153411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.153413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.153760 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.153878 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.154223 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.154466 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":66}
I20260811 12:16:08.159757 18015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Bootstrap starting.
I20260811 12:16:08.163095 18015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:08.164218 18015 log.cc:824] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:08.167438 18015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: No bootstrap required, opened a new log
I20260811 12:16:08.175212 18015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } }
I20260811 12:16:08.175647 18015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:08.175757 18015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b5602e8998c40879c210fb36631f753, State: Initialized, Role: FOLLOWER
I20260811 12:16:08.176409 18015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } }
I20260811 12:16:08.177414 18025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } } }
I20260811 12:16:08.177817 18025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:16:08.178599 18015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:16:08.188279 18048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:16:08.188455 18048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:16:08.191699 17914 master_runner.cc:428] Detected that this master 1b5602e8998c40879c210fb36631f753 is joining an existing cluster
I20260811 12:16:08.191826 17914 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.39.61:39781
I20260811 12:16:08.226303 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.201994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.7.39.62:37677)
0811 12:16:08.202185 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.202191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.209873 (+ 7682us) server_negotiation.cc:407] Connection header received
0811 12:16:08.210130 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.210149 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.210256 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.210332 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.211705 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.212477 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.214730 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.215426 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.215540 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.218856 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.218863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.218868 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.218910 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.222814 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.223856 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.223862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.223865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.224022 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.224512 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.224517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.224520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.224902 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.225029 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.225693 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.225960 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:16:08.226332 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.201668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35508 (local address 127.7.39.61:39781)
0811 12:16:08.201874 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.201880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.209256 (+ 7376us) server_negotiation.cc:407] Connection header received
0811 12:16:08.209591 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.209596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.209711 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.209803 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.211492 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.212712 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.214718 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.215440 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.215540 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.218782 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.218790 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.218794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.218830 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.222814 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.223833 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.223838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.223841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.224012 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.224491 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.224496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.224499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.224890 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.225041 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.225722 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.225951 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":46}
I20260811 12:16:08.227401 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.200839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.0.0.1:35508)
0811 12:16:08.209086 (+ 8247us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.209113 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.209403 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.210014 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.210018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.210872 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.211306 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.211320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.212920 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.212929 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.214552 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.214562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.215864 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.215871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.216153 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.217130 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.217187 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.218543 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.222996 (+ 4453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.223001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.223006 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.223692 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.224155 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.224160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.224163 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.224359 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.225103 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.225108 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.225378 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.226605 (+ 1227us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.226929 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8069,"mutex_wait_us":121,"spinlock_wait_cycles":9600,"thread_start_us":8541,"threads_started":1}
I20260811 12:16:08.228073 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.202356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:48770)
0811 12:16:08.209766 (+ 7410us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.209783 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.209975 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.210566 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.210570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.211190 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.211543 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.211553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.212645 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.212649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.214523 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.214541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.215885 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.215891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.216190 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.217068 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.217099 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.218543 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.222996 (+ 4453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.223002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.223006 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.223692 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.224165 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.224169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.224172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.224359 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.225104 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.225108 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.225378 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.226981 (+ 1603us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.227352 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7253,"mutex_wait_us":170,"spinlock_wait_cycles":117632,"thread_start_us":6368,"threads_started":1}
WARNING: no policy specified for oryx/127.7.39.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.60@KRBTEST.COM" created.
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.62@KRBTEST.COM for oryx/127.7.39.61@KRBTEST.COM
I20260811 12:16:08.257272 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.238774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35522 (local address 127.7.39.61:39781)
0811 12:16:08.238994 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.239001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.239787 (+ 786us) server_negotiation.cc:407] Connection header received
0811 12:16:08.240194 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.240198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.240299 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.240390 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.241732 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.242451 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.243865 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.244394 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.244537 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.248760 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.248769 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.248774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.248822 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.252908 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.254497 (+ 1589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.254505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.254510 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.254722 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.255251 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.255257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.255261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.255750 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.255924 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.256557 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.256840 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:16:08.257302 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.237763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.0.0.1:35522)
0811 12:16:08.239609 (+ 1846us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.239664 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.239951 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.240498 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.240506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.241292 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.241552 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.241562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.242618 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.242622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.243696 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.243717 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.244600 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.244606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.244820 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.245543 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.245563 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.248454 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.253345 (+ 4891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.253375 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.253385 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.254309 (+ 924us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.254882 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.254888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.254891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.255091 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.255927 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.255938 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.256184 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.256389 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.256847 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1584,"thread_start_us":1413,"threads_started":1}
I20260811 12:16:08.260335 17134 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.39.61:39781: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1b5602e8998c40879c210fb36631f753"
member_type: NON_VOTER
last_known_addr {
host: "127.7.39.61"
port: 39781
}
attrs {
promote: true
}
}
dest_uuid: "136c6fa878384eb88036b9fa7c3377b2"
cas_config_opid_index: -1
I20260811 12:16:08.262251 17134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } }
I20260811 12:16:08.266746 18070 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1b5602e8998c40879c210fb36631f753 (127.7.39.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } }
I20260811 12:16:08.268281 17974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:08.272084 18070 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.39.61:39781
I20260811 12:16:08.272274 17974 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Refusing update from remote peer 136c6fa878384eb88036b9fa7c3377b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 12:16:08.272832 18070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } }
I20260811 12:16:08.273101 18070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:08.273603 17914 master.cc:537] Master@127.7.39.61:39781 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.60.keytab.
Entry for principal oryx/127.7.39.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.60.keytab.
I20260811 12:16:08.275934 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.60:46679
--webserver_interface=127.7.39.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.60.keytab
--principal=oryx/127.7.39.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:37677,127.7.39.61:39781,127.7.39.60:46679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:16:08.276995 18067 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.004s
W20260811 12:16:08.281811 17117 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 12:16:08.291745 17914 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:08.292303 17914 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:08.292529 17914 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: stopping tablet replica
W20260811 12:16:08.294299 17117 consensus_peers.cc:487] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 -> Peer 1b5602e8998c40879c210fb36631f753 (127.7.39.61:39781): Couldn't send request to peer 1b5602e8998c40879c210fb36631f753. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 12:16:08.301334 17914 master.cc:552] Master@127.7.39.61:39781 shutdown complete.
I20260811 12:16:08.301503 17914 master_runner.cc:305] Clearing existing system tablet
I20260811 12:16:08.302024 17914 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:08.302139 17914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:08.305567 17914 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 12:16:08.307529 18075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:08.307966 17914 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:08.308486 17914 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 12:16:08.308640 17914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
uuid: "1b5602e8998c40879c210fb36631f753"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "fc0edcdd1d825fdb7382e4457f0217ab"
server_key_iv: "a434fb747dc2807e468ebc17b8992cb5"
server_key_version: "encryptionkey@0"
I20260811 12:16:08.314796 17914 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:08.322553 17914 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 12:16:08.322921 17914 log.cc:1196] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 12:16:08.323545 17914 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Deleting consensus metadata
I20260811 12:16:08.323889 17914 master_runner.cc:315] Copying system tablet from 127.7.39.62:37677
I20260811 12:16:08.327255 17914 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: tablet copy: Beginning tablet copy session from remote peer at address 127.7.39.62:37677
I20260811 12:16:08.345556 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.328459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:48774)
0811 12:16:08.329632 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.329655 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.329831 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.330265 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.330270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.330870 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.331171 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.331185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.332687 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.332693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.334367 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.334386 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.335575 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.335581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.335780 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.336600 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.336625 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.337795 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.342087 (+ 4292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.342094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.342099 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.342695 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.343253 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.343259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.343263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.343480 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.344229 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.344234 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.344464 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.344637 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.345007 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":899,"threads_started":1}
I20260811 12:16:08.345718 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.328548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.7.39.62:37677)
0811 12:16:08.328799 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.328806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.329777 (+ 971us) server_negotiation.cc:407] Connection header received
0811 12:16:08.329968 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.329974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.330084 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.330213 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.331348 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.332529 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.334691 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.335378 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.335534 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.338079 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.338087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.338091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.338149 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.341871 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.342880 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.342888 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.342892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.343100 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.343624 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.343629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.343633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.344070 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.344266 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.344956 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.345306 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 12:16:08.347325 17165 tablet_copy_service.cc:140] P 136c6fa878384eb88036b9fa7c3377b2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1b5602e8998c40879c210fb36631f753 ({username='oryx', principal='oryx/127.7.39.61@KRBTEST.COM'} at 127.0.0.1:48774)
I20260811 12:16:08.347901 17165 tablet_copy_service.cc:161] P 136c6fa878384eb88036b9fa7c3377b2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1b5602e8998c40879c210fb36631f753 at {username='oryx', principal='oryx/127.7.39.61@KRBTEST.COM'} at 127.0.0.1:48774: session id = 1b5602e8998c40879c210fb36631f753-00000000000000000000000000000000
I20260811 12:16:08.353019 17165 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 12:16:08.356362 17914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 12:16:08.364102 17914 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: tablet copy: Starting download of 0 data blocks...
I20260811 12:16:08.364614 17914 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: tablet copy: Starting download of 1 WAL segments...
I20260811 12:16:08.369305 17914 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 12:16:08.374929 17165 tablet_copy_service.cc:342] P 136c6fa878384eb88036b9fa7c3377b2: Request end of tablet copy session 1b5602e8998c40879c210fb36631f753-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.39.61@KRBTEST.COM'} at 127.0.0.1:48774
I20260811 12:16:08.375327 17165 tablet_copy_service.cc:433] P 136c6fa878384eb88036b9fa7c3377b2: ending tablet copy session 1b5602e8998c40879c210fb36631f753-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1b5602e8998c40879c210fb36631f753
I20260811 12:16:08.393615 17914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:08.396620 18084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:08.401520 18088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:08.410772 18093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:08.411077 17914 server_base.cc:1034] running on GCE node
I20260811 12:16:08.411556 17914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:08.412314 17914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:08.413551 17914 hybrid_clock.cc:648] HybridClock initialized: now 1786450568413481 us; error 53 us; skew 500 ppm
I20260811 12:16:08.413805 17914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450568, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:08.416124 17914 init.cc:377] Logged in from keytab as oryx/127.7.39.61@KRBTEST.COM (short username oryx)
I20260811 12:16:08.417567 17914 webserver.cc:460] Webserver started at http://127.7.39.61:40899/ using document root <none> and password file <none>
I20260811 12:16:08.417940 17914 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:08.418022 17914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:08.421376 17914 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:16:08.423542 18104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:08.424010 17914 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 12:16:08.424218 17914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
uuid: "1b5602e8998c40879c210fb36631f753"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "fc0edcdd1d825fdb7382e4457f0217ab"
server_key_iv: "a434fb747dc2807e468ebc17b8992cb5"
server_key_version: "encryptionkey@0"
I20260811 12:16:08.424495 17914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:08.443830 17914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:08.448323 17914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:08.476562 17914 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.61:39781
I20260811 12:16:08.476657 18178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.61:39781 every 8 connection(s)
I20260811 12:16:08.477852 17914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
I20260811 12:16:08.483898 18179 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:16:08.488154 18179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Bootstrap starting.
I20260811 12:16:08.512418 18179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:16:08.513083 18179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: Bootstrap complete.
I20260811 12:16:08.514308 18179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } }
I20260811 12:16:08.514608 18179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b5602e8998c40879c210fb36631f753, State: Initialized, Role: LEARNER
I20260811 12:16:08.514874 18179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } }
I20260811 12:16:08.515812 18184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } }
I20260811 12:16:08.516095 18179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:08.516319 18184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: This master's current role is: LEARNER
I20260811 12:16:08.525748 18204 catalog_manager.cc:1259] Loaded cluster ID: 7a252f3444ae484aa05fd7c3110b1bfe
I20260811 12:16:08.525874 18204 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: loading cluster ID for follower catalog manager: success
I20260811 12:16:08.529508 18204 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: acquiring CA information for follower catalog manager: success
I20260811 12:16:08.532507 18204 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:16:08.597707 17906 heartbeater.cc:503] Master 127.7.39.62:37677 was elected leader, sending a full tablet report...
I20260811 12:16:08.673362 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.665816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.0.0.1:35534)
0811 12:16:08.666057 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.666083 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.666251 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.666978 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.666984 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.667017 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:08.667276 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.667289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.669305 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.669310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.670879 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.670894 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.672269 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.672275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.672726 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.672776 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.672871 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.673019 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:16:08.673624 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.666032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.7.39.61:39781)
0811 12:16:08.666617 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.666621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.666642 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:16:08.666714 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.666718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.666785 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.666889 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:08.667443 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.669079 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.671059 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.672089 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.672218 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.672558 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.672931 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.673120 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":329,"threads_started":1}
I20260811 12:16:08.675200 18145 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 12:16:08.675885 18184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 136c6fa878384eb88036b9fa7c3377b2. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } }
I20260811 12:16:08.676210 18184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: This master's current role is: LEARNER
W20260811 12:16:08.710443 18073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:08.710836 18073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:08.711062 18073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:08.722136 18073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:08.722390 18073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:08.722510 18073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:08.722671 18073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:08.722767 18073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:08.736645 18073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:37677,127.7.39.61:39781,127.7.39.60:46679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.60.keytab
--principal=oryx/127.7.39.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.60:46679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.39.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:08.738345 18073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:08.739982 18073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:08.756273 18214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:08.756906 18215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:08.756976 18218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:08.758709 18073 server_base.cc:1034] running on GCE node
I20260811 12:16:08.759783 18073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:08.761396 18073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:08.762619 18073 hybrid_clock.cc:648] HybridClock initialized: now 1786450568762563 us; error 55 us; skew 500 ppm
I20260811 12:16:08.763006 18073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450568, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:08.767370 18073 init.cc:377] Logged in from keytab as oryx/127.7.39.60@KRBTEST.COM (short username oryx)
I20260811 12:16:08.769640 18073 webserver.cc:460] Webserver started at http://127.7.39.60:33091/ using document root <none> and password file <none>
I20260811 12:16:08.770548 18073 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:08.770663 18073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:08.770952 18073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:08.775383 18073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data/instance:
uuid: "68ee124b36cd40aa9df52c75d2662f24"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "87ea96f324ac4aac0f99aa3a857d4def"
server_key_iv: "6dbb5a9b6e53bb3c4122fba6e652bbdf"
server_key_version: "encryptionkey@0"
I20260811 12:16:08.776391 18073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal/instance:
uuid: "68ee124b36cd40aa9df52c75d2662f24"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "87ea96f324ac4aac0f99aa3a857d4def"
server_key_iv: "6dbb5a9b6e53bb3c4122fba6e652bbdf"
server_key_version: "encryptionkey@0"
I20260811 12:16:08.784477 18073 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.001s
I20260811 12:16:08.789947 18225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:08.791642 18073 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:16:08.791821 18073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
uuid: "68ee124b36cd40aa9df52c75d2662f24"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "87ea96f324ac4aac0f99aa3a857d4def"
server_key_iv: "6dbb5a9b6e53bb3c4122fba6e652bbdf"
server_key_version: "encryptionkey@0"
I20260811 12:16:08.792160 18073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:08.808053 18073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:08.814327 18073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:08.814735 18073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:08.859927 18073 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.60:46679
I20260811 12:16:08.859977 18302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.60:46679 every 8 connection(s)
I20260811 12:16:08.861660 18073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
I20260811 12:16:08.866551 18303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:08.869690 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 18073
I20260811 12:16:08.875774 18303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.60@KRBTEST.COM
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450568, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.60@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:08.902798 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.880556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.7.39.62:37677)
0811 12:16:08.881533 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.881541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.882964 (+ 1423us) server_negotiation.cc:407] Connection header received
0811 12:16:08.883163 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.883167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.883237 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.883347 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.885421 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.886452 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.888203 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.888957 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.889119 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.894195 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.894204 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.894209 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.894256 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.898338 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.899663 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.899668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.899670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.899948 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.900472 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.900477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.900479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.900853 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.901012 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.901979 (+ 967us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.902246 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":675,"threads_started":1}
I20260811 12:16:08.903450 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.872221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.7.39.60:46679)
0811 12:16:08.873857 (+ 1636us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.873870 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.873915 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:16:08.876678 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.876708 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.877302 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.877767 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.880412 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.883546 (+ 3134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.885313 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.886181 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.886339 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.892752 (+ 6413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.892768 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.892790 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.892843 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.897574 (+ 4731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.898815 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.898825 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.898829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.899134 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.899635 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.899646 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.899650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.900598 (+ 948us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.900826 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.901513 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.902086 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":1336,"thread_start_us":976,"threads_started":1}
I20260811 12:16:08.903467 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.880488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:48776)
0811 12:16:08.882765 (+ 2277us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.882837 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.883034 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.883422 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.883430 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.884677 (+ 1247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.885253 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.885270 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.886620 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.886624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.888029 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.888043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.889273 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.889280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.889435 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.890355 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.890380 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.893955 (+ 3575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.898567 (+ 4612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.898578 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.898589 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.899510 (+ 921us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.900069 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.900074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.900077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.900318 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.901048 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.901054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.901326 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.901611 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.902195 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1958,"mutex_wait_us":177,"thread_start_us":1596,"threads_started":1}
I20260811 12:16:08.905354 18303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450568, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.60@KRBTEST.COM for oryx/127.7.39.61@KRBTEST.COM
I20260811 12:16:08.923341 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.906418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.7.39.61:39781)
0811 12:16:08.906645 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.906651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.907041 (+ 390us) server_negotiation.cc:407] Connection header received
0811 12:16:08.907218 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.907224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.907314 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.907407 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.908402 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.909415 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.911048 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.911727 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.911838 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.915967 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.915972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.915976 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.916014 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.919809 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.920702 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.920707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.920709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.921021 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.921746 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.921750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.921752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.922192 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.922324 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.922714 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.923014 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:16:08.923368 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.906580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.0.0.1:35548)
0811 12:16:08.906931 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.906953 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.907078 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.907439 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.907442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.907990 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.908233 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.908243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.909562 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.909566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.910900 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.910915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.911925 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.911929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.912236 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.913030 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.913050 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.915734 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.919985 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.919990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.919993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.920543 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.921239 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.921243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.921246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.921619 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.922295 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.922298 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.922470 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.922623 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.923009 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 12:16:08.924585 18303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:16:08 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450568, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.60@KRBTEST.COM for oryx/127.7.39.60@KRBTEST.COM
I20260811 12:16:08.941069 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.925479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:46679 (local address 127.0.0.1:54116)
0811 12:16:08.925694 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:08.925713 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:08.925830 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:08.926171 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:08.926174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:08.926687 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:08.926874 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.926884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.927937 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.927941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.929229 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:08.929244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.930205 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.930210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:08.930401 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.931064 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:08.931083 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:08.933926 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:08.938293 (+ 4367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.938298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.938301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.938943 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.939331 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:08.939335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:08.939337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:08.939641 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:08.940220 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:08.940224 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:08.940361 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 12:16:08.940481 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 12:16:08.940719 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 12:16:08.941313 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:08.925449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54116 (local address 127.7.39.60:46679)
0811 12:16:08.925642 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:08.925648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:08.925773 (+ 125us) server_negotiation.cc:407] Connection header received
0811 12:16:08.925929 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:08.925934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:08.925998 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:08.926088 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:08.927015 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.927793 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.929357 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:08.930020 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:08.930153 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:08.934225 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:08.934231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:08.934234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:08.934268 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:08.938144 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.939068 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.939072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.939075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.939246 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:08.939735 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:08.939738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:08.939740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:08.940112 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:08.940240 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:08.940596 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 12:16:08.940894 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":54}
I20260811 12:16:08.947733 18303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Bootstrap starting.
I20260811 12:16:08.952440 18303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:08.954391 18303 log.cc:824] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:08.958578 18303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: No bootstrap required, opened a new log
I20260811 12:16:08.969411 18303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } }
I20260811 12:16:08.969833 18303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:08.970000 18303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ee124b36cd40aa9df52c75d2662f24, State: Initialized, Role: FOLLOWER
I20260811 12:16:08.970739 18303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } }
I20260811 12:16:08.972127 18315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } } }
I20260811 12:16:08.972908 18315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:16:08.973600 18303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:16:08.988855 18329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:16:08.989053 18329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:16:08.993221 18073 master_runner.cc:428] Detected that this master 68ee124b36cd40aa9df52c75d2662f24 is joining an existing cluster
I20260811 12:16:08.993418 18073 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.39.60:46679
I20260811 12:16:09.012524 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 17250
I20260811 12:16:09.043061 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.000833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54128 (local address 127.7.39.60:46679)
0811 12:16:09.001014 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.001020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.011609 (+ 10589us) server_negotiation.cc:407] Connection header received
0811 12:16:09.011938 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.011944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.012037 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.012107 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.015656 (+ 3549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.017317 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.020312 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.021293 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.021403 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.032608 (+ 11205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.032616 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.032621 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.032661 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.038609 (+ 5948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.039714 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.039720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.039723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.039966 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.040527 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.040532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.040536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.041225 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.041369 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.042058 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.042608 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":274,"server-negotiator.queue_time_us":54}
Aug 11 12:16:09 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.1@KRBTEST.COM
I20260811 12:16:09.044903 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.001641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:46679 (local address 127.0.0.1:54128)
0811 12:16:09.011485 (+ 9844us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.011510 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.011764 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.012363 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.012367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.013715 (+ 1348us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:09.015480 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.015499 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.017709 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.017714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.020159 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.020175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.022078 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.022089 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.022376 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.027942 (+ 5566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:09.027973 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:09.032411 (+ 4438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:09.038855 (+ 6444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.038860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.038863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.039577 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.040109 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.040113 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.040117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.040375 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.041445 (+ 1070us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:09.041450 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:09.041708 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.041913 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.042405 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9582,"thread_start_us":9007,"threads_started":1}
I20260811 12:16:09.049206 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.001681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:48792)
0811 12:16:09.015601 (+ 13920us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.015624 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.015899 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.016603 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.016609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.018305 (+ 1696us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:09.019137 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.019154 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.021095 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.021101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.025388 (+ 4287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.025406 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.027184 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.027192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.027442 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.029457 (+ 2015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:09.029482 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:09.033432 (+ 3950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:09.042707 (+ 9275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.042712 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.042716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.043239 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.043789 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.043794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.043798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.044892 (+ 1094us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.045763 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:09.045769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:09.047730 (+ 1961us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.047939 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.048292 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13697,"thread_start_us":8498,"threads_started":1}
I20260811 12:16:09.049938 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.000618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.7.39.62:37677)
0811 12:16:09.000848 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.000852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.015774 (+ 14922us) server_negotiation.cc:407] Connection header received
0811 12:16:09.016108 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.016115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.016231 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.016322 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.019404 (+ 3082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.020813 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.025640 (+ 4827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.026425 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.026551 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.033752 (+ 7201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.033760 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.033765 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.033811 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.038667 (+ 4856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.043398 (+ 4731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.043404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.043408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.043635 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.045047 (+ 1412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.045052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.045056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.045530 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.045689 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.049007 (+ 3318us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.049311 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 12:16:09.051292 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.000417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35564 (local address 127.7.39.61:39781)
0811 12:16:09.000627 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.000634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.022926 (+ 22292us) server_negotiation.cc:407] Connection header received
0811 12:16:09.023249 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.023255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.023343 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.023413 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.026496 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.027545 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.032306 (+ 4761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.033023 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.033145 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.038949 (+ 5804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.038956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.038968 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.039013 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.047240 (+ 8227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.048162 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.048169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.048172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.048407 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.049022 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.049028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.049031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.049543 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.049690 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.050546 (+ 856us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.050935 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:16:09.052723 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.002244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.0.0.1:35564)
0811 12:16:09.022799 (+ 20555us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.022838 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.023071 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.023640 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.023645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.025102 (+ 1457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:09.026103 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.026119 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.027721 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.027725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.032120 (+ 4395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.032137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.036442 (+ 4305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.036455 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.036759 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.037550 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:09.037574 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:09.038676 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:09.047396 (+ 8720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.047401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.047405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.047970 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.048578 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.048583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.048587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.048867 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.049775 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:09.049781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:09.050176 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.051555 (+ 1379us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.051916 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20315,"thread_start_us":19222,"threads_started":1}
I20260811 12:16:09.060536 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.015021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.7.39.1:35645)
0811 12:16:09.029021 (+ 14000us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.029029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.029064 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:16:09.029123 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.029130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.029499 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.031789 (+ 2290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.032227 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.034262 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.036786 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.038184 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.047513 (+ 9329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.047930 (+ 417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.047950 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.047980 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.048019 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.054777 (+ 6758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.056034 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.056045 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.056048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.056344 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.056928 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.056934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.056938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.057598 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.059389 (+ 1791us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.059574 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.059883 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13756,"thread_start_us":13462,"threads_started":1}
I20260811 12:16:09.062563 17464 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 11 12:16:09 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.62@KRBTEST.COM for oryx/127.7.39.60@KRBTEST.COM
W20260811 12:16:09.068202 18233 tablet.cc:2369] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:16:09.076014 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.056919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:46679 (local address 127.0.0.1:54132)
0811 12:16:09.057299 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.057315 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.057427 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.057820 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.057825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.058272 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:09.058714 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.058725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.060190 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.060194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.061874 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.061889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.063357 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.063381 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.063611 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.064348 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:09.064374 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:09.067714 (+ 3340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:09.072698 (+ 4984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.072705 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.072709 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.073330 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.073767 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.073773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.073777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.074055 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.074856 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:09.074861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:09.075101 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.075288 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.075590 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278}
I20260811 12:16:09.077126 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.056944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.7.39.60:46679)
0811 12:16:09.057119 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.057125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.057384 (+ 259us) server_negotiation.cc:407] Connection header received
0811 12:16:09.057575 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.057580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.057685 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.057767 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.058865 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.060011 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.062060 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.063001 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.063119 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.067930 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.067935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.067938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.067985 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.072505 (+ 4520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.073500 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.073505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.073507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.073641 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.074234 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.074238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.074240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.074720 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.074876 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.076464 (+ 1588us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.076679 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:16:09.078786 17134 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.39.60:46679: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24"
member_type: NON_VOTER
last_known_addr {
host: "127.7.39.60"
port: 46679
}
attrs {
promote: true
}
}
dest_uuid: "136c6fa878384eb88036b9fa7c3377b2"
cas_config_opid_index: 5
I20260811 12:16:09.080003 17134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } }
I20260811 12:16:09.083709 18070 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 68ee124b36cd40aa9df52c75d2662f24 (127.7.39.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.086295 18145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 LEARNER]: Refusing update from remote peer 136c6fa878384eb88036b9fa7c3377b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 12:16:09.088460 18267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:09.089866 18347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 12:16:09.092585 18070 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.39.60:46679
I20260811 12:16:09.093868 18145 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 68ee124b36cd40aa9df52c75d2662f24 (127.7.39.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.093024 18070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.097079 18070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:09.100268 18184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: NON_VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.100592 18184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: This master's current role is: LEARNER
I20260811 12:16:09.102160 18070 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: attempting to promote NON_VOTER 1b5602e8998c40879c210fb36631f753 to VOTER
I20260811 12:16:09.103428 18070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } }
I20260811 12:16:09.104200 18073 master.cc:537] Master@127.7.39.60:46679 shutting down...
I20260811 12:16:09.106613 18145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 LEARNER]: Refusing update from remote peer 136c6fa878384eb88036b9fa7c3377b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 12:16:09.107400 18267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Refusing update from remote peer 136c6fa878384eb88036b9fa7c3377b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 12:16:09.119092 18073 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:09.119660 18073 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:09.119953 18073 tablet_replica.cc:330] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: stopping tablet replica
I20260811 12:16:09.109387 18349 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
W20260811 12:16:09.123383 18226 connection.cc:441] server connection from 127.0.0.1:54132 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54132 (request call id 3) could send its response
W20260811 12:16:09.125883 17118 consensus_peers.cc:487] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 -> Peer 68ee124b36cd40aa9df52c75d2662f24 (127.7.39.60:46679): Couldn't send request to peer 68ee124b36cd40aa9df52c75d2662f24. Status: Network error: recv got EOF from 127.7.39.60:46679 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260811 12:16:09.127161 18347 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1b5602e8998c40879c210fb36631f753 (127.7.39.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.128938 18073 master.cc:552] Master@127.7.39.60:46679 shutdown complete.
I20260811 12:16:09.129176 18073 master_runner.cc:305] Clearing existing system tablet
I20260811 12:16:09.129628 18073 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:09.129778 18073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:09.131744 18349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.132254 18349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:09.131831 18347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.132551 18347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:09.132995 18145 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1b5602e8998c40879c210fb36631f753 (127.7.39.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.137334 18184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.137606 18184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:16:09.140259 18073 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.004s
I20260811 12:16:09.142143 18355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:09.142529 18073 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:09.142796 18073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:16:09.142904 18073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
uuid: "68ee124b36cd40aa9df52c75d2662f24"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "87ea96f324ac4aac0f99aa3a857d4def"
server_key_iv: "6dbb5a9b6e53bb3c4122fba6e652bbdf"
server_key_version: "encryptionkey@0"
I20260811 12:16:09.148253 18073 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:09.155699 18073 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 12:16:09.155895 18073 log.cc:1196] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 12:16:09.156399 18073 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Deleting consensus metadata
I20260811 12:16:09.156673 18073 master_runner.cc:315] Copying system tablet from 127.7.39.62:37677
I20260811 12:16:09.163436 18073 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: tablet copy: Beginning tablet copy session from remote peer at address 127.7.39.62:37677
I20260811 12:16:09.186520 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.164651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:48794)
0811 12:16:09.165662 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.165684 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.165864 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.166311 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.166315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.166946 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:09.167561 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.167580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.169685 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.169691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.171611 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.171631 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.173075 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.173083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.173410 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.174480 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:09.174506 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:09.176506 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:09.182537 (+ 6031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.182543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.182547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.183129 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.183735 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.183740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.183743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.183999 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.184870 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:09.184876 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:09.185282 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.185445 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.185811 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":672,"threads_started":1}
I20260811 12:16:09.186530 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.164518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.7.39.62:37677)
0811 12:16:09.164877 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.164884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.165809 (+ 925us) server_negotiation.cc:407] Connection header received
0811 12:16:09.166014 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.166020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.166129 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.166256 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.167769 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.169441 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.171845 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.172802 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.172969 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.176782 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.176790 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.176795 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.176848 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.182362 (+ 5514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.183276 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.183283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.183286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.183583 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.184149 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.184155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.184159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.184705 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.184877 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.185629 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.185923 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260811 12:16:09.187810 17165 tablet_copy_service.cc:140] P 136c6fa878384eb88036b9fa7c3377b2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 68ee124b36cd40aa9df52c75d2662f24 ({username='oryx', principal='oryx/127.7.39.60@KRBTEST.COM'} at 127.0.0.1:48794)
I20260811 12:16:09.188103 17165 tablet_copy_service.cc:161] P 136c6fa878384eb88036b9fa7c3377b2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 68ee124b36cd40aa9df52c75d2662f24 at {username='oryx', principal='oryx/127.7.39.60@KRBTEST.COM'} at 127.0.0.1:48794: session id = 68ee124b36cd40aa9df52c75d2662f24-00000000000000000000000000000000
I20260811 12:16:09.192706 17165 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 12:16:09.194913 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 17250
I20260811 12:16:09.195925 18073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 12:16:09.208892 18073 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: tablet copy: Starting download of 0 data blocks...
I20260811 12:16:09.209287 18073 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: tablet copy: Starting download of 1 WAL segments...
I20260811 12:16:09.212185 18073 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 12:16:09.216878 17165 tablet_copy_service.cc:342] P 136c6fa878384eb88036b9fa7c3377b2: Request end of tablet copy session 68ee124b36cd40aa9df52c75d2662f24-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.39.60@KRBTEST.COM'} at 127.0.0.1:48794
I20260811 12:16:09.217067 17165 tablet_copy_service.cc:433] P 136c6fa878384eb88036b9fa7c3377b2: ending tablet copy session 68ee124b36cd40aa9df52c75d2662f24-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 68ee124b36cd40aa9df52c75d2662f24
I20260811 12:16:09.218133 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:35645
--local_ip_for_outbound_sockets=127.7.39.1
--tserver_master_addrs=127.7.39.62:37677
--webserver_port=41325
--webserver_interface=127.7.39.1
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab
--principal=oryx/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781,127.7.39.60:46679 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:16:09.229364 18073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:09.232811 18365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:09.233331 18366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:09.245245 18374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:09.250679 18073 server_base.cc:1034] running on GCE node
I20260811 12:16:09.251179 18073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:09.251924 18073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:09.253145 18073 hybrid_clock.cc:648] HybridClock initialized: now 1786450569253100 us; error 38 us; skew 500 ppm
I20260811 12:16:09.253435 18073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:09 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450569, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:09.255702 18073 init.cc:377] Logged in from keytab as oryx/127.7.39.60@KRBTEST.COM (short username oryx)
I20260811 12:16:09.257104 18073 webserver.cc:460] Webserver started at http://127.7.39.60:35823/ using document root <none> and password file <none>
I20260811 12:16:09.257478 18073 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:09.257581 18073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:09.260905 18073 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:16:09.263060 18388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:09.263518 18073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260811 12:16:09.263666 18073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
uuid: "68ee124b36cd40aa9df52c75d2662f24"
format_stamp: "Formatted at 2026-08-11 12:16:08 on dist-test-slave-rmhg"
server_key: "87ea96f324ac4aac0f99aa3a857d4def"
server_key_iv: "6dbb5a9b6e53bb3c4122fba6e652bbdf"
server_key_version: "encryptionkey@0"
I20260811 12:16:09.263872 18073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:09.304235 18073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:09.308735 18073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:09.336130 18073 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.60:46679
I20260811 12:16:09.336192 18477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.60:46679 every 8 connection(s)
I20260811 12:16:09.337411 18073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
I20260811 12:16:09.342787 18481 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:16:09.346843 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Bootstrap starting.
I20260811 12:16:09.367874 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:16:09.368443 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Bootstrap complete.
I20260811 12:16:09.369627 18481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } }
I20260811 12:16:09.369944 18481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 68ee124b36cd40aa9df52c75d2662f24, State: Initialized, Role: LEARNER
I20260811 12:16:09.370222 18481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } }
I20260811 12:16:09.371202 18485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.371521 18481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:09.371742 18485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: This master's current role is: LEARNER
I20260811 12:16:09.383515 18506 catalog_manager.cc:1259] Loaded cluster ID: 7a252f3444ae484aa05fd7c3110b1bfe
I20260811 12:16:09.383723 18506 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: loading cluster ID for follower catalog manager: success
I20260811 12:16:09.387385 18506 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: acquiring CA information for follower catalog manager: success
I20260811 12:16:09.390156 18506 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:16:09.631065 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.622998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54148 (local address 127.7.39.60:46679)
0811 12:16:09.623752 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.623757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.623775 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:16:09.623840 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.623843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.623914 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.624023 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:09.624525 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.625980 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.628694 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.629790 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.629904 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.630077 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.630495 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.630602 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":507,"threads_started":1}
I20260811 12:16:09.631147 18508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.622813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:46679 (local address 127.0.0.1:54148)
0811 12:16:09.623435 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.623458 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.623627 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.624070 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.624076 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.624105 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:09.624336 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.624346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.626221 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.626228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.628510 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.628533 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.630030 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.630040 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.630326 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.630369 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.630476 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.630651 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":373,"threads_started":1}
I20260811 12:16:09.632347 18435 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 12:16:09.633107 18485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 136c6fa878384eb88036b9fa7c3377b2. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:09.633433 18485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: This master's current role is: LEARNER
W20260811 12:16:09.699965 18363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:09.700407 18363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:09.700649 18363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:09.710237 18363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:09.710436 18363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:09.710650 18363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:09.723505 18363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab
--principal=oryx/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:35645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=41325
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781,127.7.39.60:46679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:09.725322 18363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:09.727084 18363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:09.744635 18516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:09.745217 18517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:09.745288 18520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:09.746452 18363 server_base.cc:1034] running on GCE node
I20260811 12:16:09.747094 18363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:09.748288 18363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:09.749621 18363 hybrid_clock.cc:648] HybridClock initialized: now 1786450569749483 us; error 50 us; skew 500 ppm
I20260811 12:16:09.750069 18363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:09 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450569, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:09.762665 18363 init.cc:377] Logged in from keytab as oryx/127.7.39.1@KRBTEST.COM (short username oryx)
I20260811 12:16:09.764837 18363 webserver.cc:460] Webserver started at http://127.7.39.1:41325/ using document root <none> and password file <none>
I20260811 12:16:09.765913 18363 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:09.766131 18363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:09.772670 18363 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:16:09.776934 18534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:09.778642 18363 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 12:16:09.778826 18363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "9821077cf82e4c0c941244fbee6abf1e"
format_stamp: "Formatted at 2026-08-11 12:16:06 on dist-test-slave-rmhg"
server_key: "6f37fa804df23b53e13eacff89994f63"
server_key_iv: "c9262b205eef637cddaf6b03671fc346"
server_key_version: "encryptionkey@0"
I20260811 12:16:09.780406 18363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:09.811262 18363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:09.816988 18363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:09.817425 18363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:09.818904 18363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:09.820978 18363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:09.821107 18363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:09.821246 18363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:09.821379 18363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:09.891727 18363 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:35645
I20260811 12:16:09.891883 18719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:35645 every 8 connection(s)
I20260811 12:16:09.893787 18363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:09.902258 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 18363
I20260811 12:16:09.902810 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 17488
Aug 11 12:16:09 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.2@KRBTEST.COM
Aug 11 12:16:09 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450569, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
Aug 11 12:16:09 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450569, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.60@KRBTEST.COM
Aug 11 12:16:09 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450569, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.61@KRBTEST.COM
I20260811 12:16:09.960119 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.898164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:33187 (local address 127.7.39.61:39781)
0811 12:16:09.929577 (+ 31413us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.929585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.929636 (+ 51us) server_negotiation.cc:407] Connection header received
0811 12:16:09.930754 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.930760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.930858 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.930973 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.934342 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.935786 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.939510 (+ 3724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.940465 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.940581 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.950702 (+ 10121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.950711 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.950716 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.950760 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.954850 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.955741 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.955748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.955751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.955940 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.956699 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.956705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.956708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.958130 (+ 1422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.958237 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.959410 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.959656 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31263,"thread_start_us":31473,"threads_started":1}
I20260811 12:16:09.961325 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.897896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:34001 (local address 127.7.39.60:46679)
0811 12:16:09.898647 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.898654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.932095 (+ 33441us) server_negotiation.cc:407] Connection header received
0811 12:16:09.932450 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.932457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.932551 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.932639 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.934979 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.936228 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.939441 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.940347 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.940461 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.948017 (+ 7556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.948025 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.948029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.948076 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.953186 (+ 5110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.955109 (+ 1923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.955116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.955119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.955312 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.959095 (+ 3783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.959100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.959104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.959469 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.959640 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.960269 (+ 629us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.960665 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586}
I20260811 12:16:09.962637 18727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.899421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:37455 (local address 127.7.39.62:37677)
0811 12:16:09.922514 (+ 23093us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.922522 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.928090 (+ 5568us) server_negotiation.cc:407] Connection header received
0811 12:16:09.930655 (+ 2565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.930663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.930782 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.930907 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.934451 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.935839 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.939442 (+ 3603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.940379 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.940471 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.946824 (+ 6353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.946832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.946837 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.946885 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.957261 (+ 10376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.959795 (+ 2534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.959802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.959806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.960033 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.960512 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.960517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.960520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.960980 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.961234 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.961847 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.962083 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22928,"thread_start_us":22831,"threads_started":1}
I20260811 12:16:09.963910 18731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.898627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.7.39.1:33187)
0811 12:16:09.926098 (+ 27471us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.926168 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.930271 (+ 4103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.931259 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.931271 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.932396 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:09.934088 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.934122 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.936052 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.936057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.939303 (+ 3246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.939320 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.941544 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.941555 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.941924 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.942772 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:09.942796 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:09.950347 (+ 7551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:09.955016 (+ 4669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.955022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.955025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.955573 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.956100 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.956105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.956109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.956557 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.958319 (+ 1762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:09.958325 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:09.958838 (+ 513us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.960500 (+ 1662us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.961191 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26752,"mutex_wait_us":283,"thread_start_us":28821,"threads_started":1}
I20260811 12:16:09.963940 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.899281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:37455)
0811 12:16:09.927855 (+ 28574us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.927887 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.930276 (+ 2389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.931258 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.931271 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.933091 (+ 1820us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:09.934232 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.934248 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.936052 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.936057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.939241 (+ 3184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.939259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.940978 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.940985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.941431 (+ 446us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.942503 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:09.942542 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:09.946481 (+ 3939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:09.958093 (+ 11612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.958099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.958103 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.958785 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.960186 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.960191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.960194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.960339 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.961127 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:09.961131 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:09.961331 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.961497 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.961795 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28247,"mutex_wait_us":360,"thread_start_us":30868,"threads_started":1}
I20260811 12:16:09.964334 18728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.899651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:46679 (local address 127.7.39.1:34001)
0811 12:16:09.931968 (+ 32317us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:09.931991 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:09.932267 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:09.932887 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:09.932892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:09.934329 (+ 1437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:09.934789 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.934802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.936430 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.936435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.939241 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:09.939259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.940944 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.940958 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:09.941435 (+ 477us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.944415 (+ 2980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:09.944463 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:09.947610 (+ 3147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:09.953380 (+ 5770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.953392 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.953416 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.954773 (+ 1357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.958745 (+ 3972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:09.958750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:09.958753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:09.958956 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:09.959733 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:09.959738 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:09.959892 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 12:16:09.961617 (+ 1725us) client_negotiation.cc:240] Negotiation successful
0811 12:16:09.962486 (+ 869us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32168,"mutex_wait_us":895,"spinlock_wait_cycles":23936,"thread_start_us":23279,"threads_started":1}
I20260811 12:16:09.966050 18722 heartbeater.cc:347] Connected to a master server at 127.7.39.61:39781
I20260811 12:16:09.966226 18720 heartbeater.cc:347] Connected to a master server at 127.7.39.60:46679
I20260811 12:16:09.966423 18722 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:09.966648 18721 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37677
I20260811 12:16:09.966782 18721 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:09.967002 18720 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:09.967555 18735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:09.904190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.7.39.2:37651)
0811 12:16:09.924812 (+ 20622us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:09.924822 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:09.924858 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:16:09.924940 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:09.924949 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:09.925262 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:09.925460 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:09.928681 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.930821 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.933293 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:09.934498 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:09.934628 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:09.945435 (+ 10807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:09.945457 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:09.945478 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:09.945555 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:09.958541 (+ 12986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.961856 (+ 3315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.961868 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.961872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.962221 (+ 349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:09.963275 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:09.963281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:09.963284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:09.963846 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:09.966336 (+ 2490us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:09.966635 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 12:16:09.967021 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20308,"thread_start_us":17686,"threads_started":1}
I20260811 12:16:09.968215 18722 heartbeater.cc:511] Master 127.7.39.61:39781 requested a full tablet report, sending...
I20260811 12:16:09.968278 18720 heartbeater.cc:511] Master 127.7.39.60:46679 requested a full tablet report, sending...
I20260811 12:16:09.968412 18721 heartbeater.cc:511] Master 127.7.39.62:37677 requested a full tablet report, sending...
I20260811 12:16:09.969764 17675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:09.971167 18420 ts_manager.cc:194] Registered new tserver with Master: 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645)
I20260811 12:16:09.971115 18129 ts_manager.cc:194] Registered new tserver with Master: 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645)
I20260811 12:16:09.972406 17134 ts_manager.cc:194] Re-registered known tserver with Master: 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645)
I20260811 12:16:09.973572 17134 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:37455
I20260811 12:16:10.032265 18347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: NON_VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: true } } }
I20260811 12:16:10.032835 18347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:10.034603 18737 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: attempting to promote NON_VOTER 68ee124b36cd40aa9df52c75d2662f24 to VOTER
I20260811 12:16:10.035239 18737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } }
I20260811 12:16:10.037266 18145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Refusing update from remote peer 136c6fa878384eb88036b9fa7c3377b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 12:16:10.037446 18435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 LEARNER]: Refusing update from remote peer 136c6fa878384eb88036b9fa7c3377b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 12:16:10.037943 18347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 12:16:10.038492 18739 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 12:16:10.041416 18347 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68ee124b36cd40aa9df52c75d2662f24 (127.7.39.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:10.044273 18435 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68ee124b36cd40aa9df52c75d2662f24 (127.7.39.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:10.046147 18739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:10.046656 18739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:10.046406 18145 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68ee124b36cd40aa9df52c75d2662f24 (127.7.39.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:10.049258 18485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:10.049523 18485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:16:10.050755 18739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:10.051017 18739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:10.045569 18347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:10.051762 18347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:10.052515 18747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "136c6fa878384eb88036b9fa7c3377b2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:10.053248 18747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:16:10.069326 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 17488
I20260811 12:16:10.093235 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:37651
--local_ip_for_outbound_sockets=127.7.39.2
--tserver_master_addrs=127.7.39.62:37677
--webserver_port=41629
--webserver_interface=127.7.39.2
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab
--principal=oryx/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781,127.7.39.60:46679 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:10.449535 18748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:10.449867 18748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:10.449995 18748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:10.457756 18748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:10.457901 18748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:10.458060 18748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:10.468537 18748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab
--principal=oryx/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:37651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=41629
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781,127.7.39.60:46679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:10.470247 18748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:10.471675 18748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:10.487958 18758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:10.488071 18765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:10.489166 18760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:10.489180 18748 server_base.cc:1034] running on GCE node
I20260811 12:16:10.489919 18748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:10.490942 18748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:10.492190 18748 hybrid_clock.cc:648] HybridClock initialized: now 1786450570492106 us; error 54 us; skew 500 ppm
I20260811 12:16:10.492522 18748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:10 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450570, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:10.495950 18748 init.cc:377] Logged in from keytab as oryx/127.7.39.2@KRBTEST.COM (short username oryx)
I20260811 12:16:10.497464 18748 webserver.cc:460] Webserver started at http://127.7.39.2:41629/ using document root <none> and password file <none>
I20260811 12:16:10.498240 18748 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:10.498400 18748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:10.504267 18748 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 12:16:10.507390 18780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:10.508891 18748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:16:10.509044 18748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "0c4039332f2847d9b9743590f94bafe1"
format_stamp: "Formatted at 2026-08-11 12:16:06 on dist-test-slave-rmhg"
server_key: "a79a596747cf1e95c76a25ba62931b53"
server_key_iv: "9bfc31c46ea32f549660534413f757f1"
server_key_version: "encryptionkey@0"
I20260811 12:16:10.510116 18748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:10.530586 18748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:10.535202 18748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:10.535494 18748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:10.536549 18748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:10.538499 18748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:10.538604 18748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:10.538785 18748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:10.538864 18748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:10.570791 18748 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:37651
I20260811 12:16:10.570861 18964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:37651 every 8 connection(s)
I20260811 12:16:10.572359 18748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:10.580173 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 18748
I20260811 12:16:10.580514 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 17691
Aug 11 12:16:10 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450570, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:10.601240 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:10.575880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:37107 (local address 127.7.39.62:37677)
0811 12:16:10.576403 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:10.576409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:10.577391 (+ 982us) server_negotiation.cc:407] Connection header received
0811 12:16:10.580426 (+ 3035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:10.580432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:10.580529 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:10.580632 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:10.583312 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.584514 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.587297 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.587995 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.588100 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:10.593821 (+ 5721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:10.593828 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:10.593833 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:10.593889 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:10.597099 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:10.598267 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:10.598273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:10.598277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:10.598469 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:10.598955 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:10.598962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:10.598965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:10.599330 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:10.599484 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:10.600567 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0811 12:16:10.600812 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":287,"threads_started":1}
Aug 11 12:16:10 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450570, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.60@KRBTEST.COM
I20260811 12:16:10.603698 18976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:10.576132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:37107)
0811 12:16:10.577214 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:10.577273 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:10.580056 (+ 2783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:10.581278 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:10.581292 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:10.582157 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:10.583067 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:10.583095 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.584774 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.584780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:10.587080 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:10.587100 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.588511 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.588518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:10.588880 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:10.589790 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:10.589824 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:10.593402 (+ 3578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:10.597282 (+ 3880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:10.597294 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:10.597318 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:10.598088 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:10.598584 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:10.598589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:10.598594 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:10.598825 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:10.599558 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:10.599566 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:10.600059 (+ 493us) client_negotiation.cc:769] Sending connection context
0811 12:16:10.601484 (+ 1425us) client_negotiation.cc:240] Negotiation successful
0811 12:16:10.602078 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":605,"threads_started":1}
I20260811 12:16:10.605556 18967 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37677
I20260811 12:16:10.605891 18967 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:10.607195 18967 heartbeater.cc:511] Master 127.7.39.62:37677 requested a full tablet report, sending...
I20260811 12:16:10.609221 17134 ts_manager.cc:194] Re-registered known tserver with Master: 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:10.610510 17134 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:37107
Aug 11 12:16:10 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450570, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.61@KRBTEST.COM
I20260811 12:16:10.615137 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:10.575390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:35605 (local address 127.7.39.60:46679)
0811 12:16:10.575974 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:10.575980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:10.593791 (+ 17811us) server_negotiation.cc:407] Connection header received
0811 12:16:10.593971 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:10.594005 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:10.594096 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:10.594210 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:10.595672 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.596859 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.598438 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.599120 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.599246 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:10.604983 (+ 5737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:10.604990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:10.604994 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:10.605038 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:10.610966 (+ 5928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:10.612261 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:10.612268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:10.612271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:10.612440 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:10.612987 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:10.612993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:10.612996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:10.613469 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:10.613683 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:10.614266 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 12:16:10.614504 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":1067,"threads_started":1}
I20260811 12:16:10.616413 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:10.576132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:46679 (local address 127.7.39.2:35605)
0811 12:16:10.593672 (+ 17540us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:10.593693 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:10.593831 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:10.594256 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:10.594261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:10.595044 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:10.595436 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:10.595450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.597035 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.597040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:10.598253 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:10.598269 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.599310 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.599317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:10.599507 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:10.600143 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:10.600165 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:10.604734 (+ 4569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:10.611160 (+ 6426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:10.611164 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:10.611168 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:10.612112 (+ 944us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:10.612598 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:10.612602 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:10.612605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:10.612843 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:10.613768 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:10.613772 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:10.613948 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 12:16:10.615435 (+ 1487us) client_negotiation.cc:240] Negotiation successful
0811 12:16:10.615804 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17361,"mutex_wait_us":402,"spinlock_wait_cycles":38016,"thread_start_us":24689,"threads_started":1}
I20260811 12:16:10.617691 18965 heartbeater.cc:347] Connected to a master server at 127.7.39.60:46679
I20260811 12:16:10.617861 18965 heartbeater.cc:464] Registering TS with master...
Aug 11 12:16:10 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.3@KRBTEST.COM
I20260811 12:16:10.618599 18965 heartbeater.cc:511] Master 127.7.39.60:46679 requested a full tablet report, sending...
I20260811 12:16:10.619751 18420 ts_manager.cc:194] Registered new tserver with Master: 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:10.623706 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:10.576722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.7.39.2:45591)
0811 12:16:10.600992 (+ 24270us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:10.601013 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:10.601141 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:10.603027 (+ 1886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:10.603032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:10.603960 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:10.604448 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:10.604463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.606549 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.606554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:10.608741 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:10.608757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.610182 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.610189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:10.610406 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:10.611616 (+ 1210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:10.611638 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:10.615488 (+ 3850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:10.620618 (+ 5130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:10.620622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:10.620625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:10.621281 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:10.621770 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:10.621774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:10.621777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:10.622049 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:10.622835 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:10.622839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:10.623000 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 12:16:10.623114 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 12:16:10.623350 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24127,"mutex_wait_us":28,"thread_start_us":16642,"threads_started":1}
I20260811 12:16:10.624030 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:10.575429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:45591 (local address 127.7.39.61:39781)
0811 12:16:10.590187 (+ 14758us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:10.590194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:10.601101 (+ 10907us) server_negotiation.cc:407] Connection header received
0811 12:16:10.601339 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:10.601345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:10.601449 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:10.601562 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:10.604656 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.606199 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.608920 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.609701 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.609807 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:10.615701 (+ 5894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:10.615707 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:10.615710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:10.615750 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:10.620450 (+ 4700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:10.621406 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:10.621412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:10.621415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:10.621580 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:10.622197 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:10.622202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:10.622207 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:10.622693 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:10.622831 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:10.623284 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 12:16:10.623500 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14606,"thread_start_us":14771,"threads_started":1}
I20260811 12:16:10.624997 18968 heartbeater.cc:347] Connected to a master server at 127.7.39.61:39781
I20260811 12:16:10.625272 18968 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:10.625893 18968 heartbeater.cc:511] Master 127.7.39.61:39781 requested a full tablet report, sending...
I20260811 12:16:10.627331 18129 ts_manager.cc:194] Registered new tserver with Master: 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:10.628816 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:10.581707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.7.39.3:42173)
0811 12:16:10.587644 (+ 5937us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:10.587654 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:10.600049 (+ 12395us) server_negotiation.cc:407] Connection header received
0811 12:16:10.600565 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:10.600576 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:10.600888 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:10.601064 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:10.605579 (+ 4515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.607553 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.609736 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:10.611010 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:10.611101 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:10.619748 (+ 8647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:10.619772 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:10.619794 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:10.619854 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:10.625482 (+ 5628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:10.626478 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:10.626489 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:10.626493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:10.626793 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:10.627233 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:10.627236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:10.627238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:10.627703 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:10.627974 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:10.628277 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 12:16:10.628502 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5753,"thread_start_us":5585,"threads_started":1}
I20260811 12:16:10.630273 17896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:10.718652 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 17691
I20260811 12:16:10.740857 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:42173
--local_ip_for_outbound_sockets=127.7.39.3
--tserver_master_addrs=127.7.39.62:37677
--webserver_port=45443
--webserver_interface=127.7.39.3
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab
--principal=oryx/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781,127.7.39.60:46679 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:16:10.976390 18721 heartbeater.cc:503] Master 127.7.39.62:37677 was elected leader, sending a full tablet report...
W20260811 12:16:11.088567 18984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:11.088861 18984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:11.088987 18984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:11.096305 18984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:11.096422 18984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:11.096578 18984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:11.106644 18984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab
--principal=oryx/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:42173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=45443
--tserver_master_addrs=127.7.39.62:37677,127.7.39.61:39781,127.7.39.60:46679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:11.108031 18984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:11.109602 18984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:11.124497 18992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:11.124540 18993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:11.124471 18995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:11.125771 18984 server_base.cc:1034] running on GCE node
I20260811 12:16:11.126449 18984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:11.127426 18984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:11.128654 18984 hybrid_clock.cc:648] HybridClock initialized: now 1786450571128567 us; error 52 us; skew 500 ppm
I20260811 12:16:11.129060 18984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:11 dist-test-slave-rmhg krb5kdc[17064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450571, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:11.132296 18984 init.cc:377] Logged in from keytab as oryx/127.7.39.3@KRBTEST.COM (short username oryx)
I20260811 12:16:11.133886 18984 webserver.cc:460] Webserver started at http://127.7.39.3:45443/ using document root <none> and password file <none>
I20260811 12:16:11.134601 18984 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:11.134742 18984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:11.140482 18984 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:16:11.143482 19008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:11.145116 18984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:16:11.145337 18984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "9a2cc2bb2de3418daf003e6958c41172"
format_stamp: "Formatted at 2026-08-11 12:16:07 on dist-test-slave-rmhg"
server_key: "5b107993ad306ac89c33d27128799513"
server_key_iv: "ae626073cd104594debed7eaa900a0fb"
server_key_version: "encryptionkey@0"
I20260811 12:16:11.146382 18984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:11.170845 18984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:11.176213 18984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:11.176524 18984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:11.177762 18984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:11.179374 18984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:11.179498 18984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:11.179610 18984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:11.179672 18984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:11.213694 18984 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:42173
I20260811 12:16:11.213788 19183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:42173 every 8 connection(s)
I20260811 12:16:11.215238 18984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:11.218452 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 18984
Aug 11 12:16:11 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450571, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.61@KRBTEST.COM
Aug 11 12:16:11 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450571, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
Aug 11 12:16:11 dist-test-slave-rmhg krb5kdc[17064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450571, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.60@KRBTEST.COM
I20260811 12:16:11.254717 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.219204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:47815 (local address 127.7.39.61:39781)
0811 12:16:11.220310 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.220318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.228088 (+ 7770us) server_negotiation.cc:407] Connection header received
0811 12:16:11.228332 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.228339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.228427 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.228535 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.230015 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.230861 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.233561 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.234263 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.234384 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.242450 (+ 8066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.242460 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.242465 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.242514 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.247823 (+ 5309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.249920 (+ 2097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.249927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.249931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.250221 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.250704 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.250710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.250713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.251285 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.251437 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.253091 (+ 1654us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.253438 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":963,"thread_start_us":855,"threads_started":1}
I20260811 12:16:11.254771 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.219644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:43569 (local address 127.7.39.62:37677)
0811 12:16:11.225628 (+ 5984us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.225635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.227286 (+ 1651us) server_negotiation.cc:407] Connection header received
0811 12:16:11.227358 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.227362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.227429 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.227523 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.230423 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.231380 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.233493 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.234111 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.234242 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.240654 (+ 6412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.240663 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.240667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.240713 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.249705 (+ 8992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.250653 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.251471 (+ 818us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.251475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.251729 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.252235 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.252241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.252244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.252753 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.252914 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.253925 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.254197 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5830,"thread_start_us":5703,"threads_started":1}
I20260811 12:16:11.254781 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.220963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:43569)
0811 12:16:11.227062 (+ 6099us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.227084 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.227217 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.227573 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.227577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.229893 (+ 2316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:11.230186 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.230198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.231530 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.231535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.233337 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.233353 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.234644 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.234650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.234832 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.235769 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:11.235804 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:11.240323 (+ 4519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:11.249846 (+ 9523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:11.249852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:11.249856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:11.250529 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:11.251855 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:11.251860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:11.251863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:11.252122 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:11.252871 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:11.252875 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:11.253011 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.253234 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.253503 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5975,"mutex_wait_us":72,"thread_start_us":4860,"threads_started":1}
I20260811 12:16:11.256935 19186 heartbeater.cc:347] Connected to a master server at 127.7.39.62:37677
I20260811 12:16:11.257390 19186 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:11.258787 19186 heartbeater.cc:511] Master 127.7.39.62:37677 requested a full tablet report, sending...
I20260811 12:16:11.261034 17134 ts_manager.cc:194] Re-registered known tserver with Master: 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173)
I20260811 12:16:11.262564 17134 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:43569
I20260811 12:16:11.267206 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.218865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.7.39.3:47815)
0811 12:16:11.227963 (+ 9098us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.227985 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.228210 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.228566 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.228570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.229071 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:11.229739 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.229768 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.231032 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.231037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.233441 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.233452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.234483 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.234491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.234839 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.236082 (+ 1243us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:11.236103 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:11.242165 (+ 6062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:11.248013 (+ 5848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:11.248026 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:11.248052 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:11.248939 (+ 887us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:11.250330 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:11.250336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:11.250339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:11.250610 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:11.251414 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:11.251423 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:11.251936 (+ 513us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.252301 (+ 365us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.252999 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7493,"mutex_wait_us":93,"thread_start_us":1317,"threads_started":1}
I20260811 12:16:11.268574 19188 heartbeater.cc:347] Connected to a master server at 127.7.39.61:39781
I20260811 12:16:11.268725 19188 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:11.269656 19188 heartbeater.cc:511] Master 127.7.39.61:39781 requested a full tablet report, sending...
I20260811 12:16:11.271258 18129 ts_manager.cc:194] Registered new tserver with Master: 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173)
I20260811 12:16:11.272684 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.219948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:60413 (local address 127.7.39.60:46679)
0811 12:16:11.226347 (+ 6399us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.226354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.226378 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:16:11.226455 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.226461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.226559 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.226707 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.231096 (+ 4389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.232162 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.234066 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.234702 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.234828 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.257872 (+ 23044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.257883 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.257888 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.257937 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.263453 (+ 5516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.265537 (+ 2084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.265544 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.265548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.265821 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.267984 (+ 2163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.267991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.267995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.268548 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.268705 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.271976 (+ 3271us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.272229 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6265,"thread_start_us":6142,"threads_started":1}
I20260811 12:16:11.273038 19193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.218865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:46679 (local address 127.7.39.3:60413)
0811 12:16:11.221214 (+ 2349us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.221269 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.224088 (+ 2819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.227070 (+ 2982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.227083 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.228602 (+ 1519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:11.229739 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.229768 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.232376 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.232382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.233891 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.233906 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.238286 (+ 4380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.238295 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.238606 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.239422 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:11.239447 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:11.256173 (+ 16726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:11.263759 (+ 7586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:11.263765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:11.263770 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:11.264676 (+ 906us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:11.266569 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:11.266575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:11.266580 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:11.266935 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:11.268820 (+ 1885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:11.268825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:11.268993 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.271853 (+ 2860us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.272357 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1987,"mutex_wait_us":3,"thread_start_us":7664,"threads_started":1}
I20260811 12:16:11.274211 19185 heartbeater.cc:347] Connected to a master server at 127.7.39.60:46679
I20260811 12:16:11.274394 19185 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:11.275128 19185 heartbeater.cc:511] Master 127.7.39.60:46679 requested a full tablet report, sending...
I20260811 12:16:11.277331 18420 ts_manager.cc:194] Registered new tserver with Master: 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173)
I20260811 12:16:11.295185 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.241955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.7.39.61:39781)
0811 12:16:11.255082 (+ 13127us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.255088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.274445 (+ 19357us) server_negotiation.cc:407] Connection header received
0811 12:16:11.274759 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.274765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.274896 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.275011 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.279309 (+ 4298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.280532 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.282637 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.283252 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.283372 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.287369 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.287376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.287380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.287426 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.291356 (+ 3930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.292262 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.292267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.292269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.292449 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.292988 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.292994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.292997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.293553 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.293872 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.294573 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.294826 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13015,"thread_start_us":13269,"threads_started":1}
I20260811 12:16:11.295277 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.233209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.7.39.62:37677)
0811 12:16:11.255098 (+ 21889us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.255103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.257450 (+ 2347us) server_negotiation.cc:407] Connection header received
0811 12:16:11.257663 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.257668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.257747 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.257829 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.274936 (+ 17107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.276421 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.279196 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.279877 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.279973 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.286283 (+ 6310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.286290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.286295 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.286440 (+ 145us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.290344 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.292661 (+ 2317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.292667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.292671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.292857 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.293434 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.293440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.293444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.293858 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.294312 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.294554 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.294911 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21763,"thread_start_us":38793,"threads_started":1}
I20260811 12:16:11.295486 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.235391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40574 (local address 127.7.39.60:46679)
0811 12:16:11.237420 (+ 2029us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.237426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.270442 (+ 33016us) server_negotiation.cc:407] Connection header received
0811 12:16:11.270802 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.270809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.270889 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.270980 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.277795 (+ 6815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.279560 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.282365 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.283272 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.283387 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.287453 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.287464 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.287469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.287512 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.291605 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.292701 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.292707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.292711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.292910 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.293369 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.293374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.293378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.293793 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.294139 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.294778 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.295018 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1874,"thread_start_us":1705,"threads_started":1}
I20260811 12:16:11.301947 17134 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:16:11.312592 17134 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:16:11.369016 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.340256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:35645 (local address 127.0.0.1:37348)
0811 12:16:11.352105 (+ 11849us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.352130 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.352348 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.353871 (+ 1523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.353876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.353908 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:11.354655 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.354673 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.356800 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.356805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.360019 (+ 3214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.360038 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.367997 (+ 7959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.368004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.368245 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.368283 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.368373 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.368533 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11633,"thread_start_us":19912,"threads_started":1}
I20260811 12:16:11.370430 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.340665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.7.39.1:35645)
0811 12:16:11.350145 (+ 9480us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.350152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.352511 (+ 2359us) server_negotiation.cc:407] Connection header received
0811 12:16:11.352697 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.352707 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.353002 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.353289 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:11.354928 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.356601 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.365549 (+ 8948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.367491 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.367601 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.367812 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.369711 (+ 1899us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.369928 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9217,"thread_start_us":9105,"threads_started":1}
I20260811 12:16:11.376539 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.341793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37651 (local address 127.0.0.1:36720)
0811 12:16:11.360662 (+ 18869us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.360686 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.360843 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.362217 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.362222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.362251 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:11.363111 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.363126 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.370789 (+ 7663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.370795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.373234 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.373254 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.375588 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.375597 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.375790 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.375823 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.375929 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.376130 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18673,"thread_start_us":9806,"threads_started":1}
I20260811 12:16:11.376717 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.342115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36720 (local address 127.7.39.2:37651)
0811 12:16:11.361517 (+ 19402us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.361527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.361556 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:16:11.361663 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.361672 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.361986 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.363372 (+ 1386us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:11.363564 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.365328 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.373499 (+ 8171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.375288 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.375469 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.375709 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.376029 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.376288 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19184,"thread_start_us":24682,"threads_started":1}
I20260811 12:16:11.379092 18616 tablet_service.cc:1467] Processing CreateTablet for tablet 672dfb00d205412c803a357ca601eec8 (DEFAULT_TABLE table=test-table [id=a9ea43ad47e74d59b205daf4c65f6ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:11.381417 18616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 672dfb00d205412c803a357ca601eec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:11.381217 18860 tablet_service.cc:1467] Processing CreateTablet for tablet 672dfb00d205412c803a357ca601eec8 (DEFAULT_TABLE table=test-table [id=a9ea43ad47e74d59b205daf4c65f6ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:11.382833 18860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 672dfb00d205412c803a357ca601eec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:11.383769 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.360626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:42173 (local address 127.0.0.1:47220)
0811 12:16:11.369377 (+ 8751us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.369397 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.369495 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.374650 (+ 5155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.374654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.374681 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:11.374932 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.374941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.379244 (+ 4303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.379250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.380912 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.380927 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.382633 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.382639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.383040 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.383077 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.383166 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.383336 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8639,"thread_start_us":13251,"threads_started":1}
I20260811 12:16:11.383773 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.360991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.7.39.3:42173)
0811 12:16:11.373101 (+ 12110us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.373113 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.373200 (+ 87us) server_negotiation.cc:407] Connection header received
0811 12:16:11.373347 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.373356 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.373735 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.374026 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:11.377093 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.379078 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.381079 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.382464 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.382594 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.382847 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.383232 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.383468 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11802,"thread_start_us":12221,"threads_started":1}
I20260811 12:16:11.387965 19080 tablet_service.cc:1467] Processing CreateTablet for tablet 672dfb00d205412c803a357ca601eec8 (DEFAULT_TABLE table=test-table [id=a9ea43ad47e74d59b205daf4c65f6ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:11.389286 19080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 672dfb00d205412c803a357ca601eec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:11.405973 19237 tablet_bootstrap.cc:492] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Bootstrap starting.
I20260811 12:16:11.408967 19235 tablet_bootstrap.cc:492] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: Bootstrap starting.
I20260811 12:16:11.411456 19236 tablet_bootstrap.cc:492] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Bootstrap starting.
I20260811 12:16:11.414191 19235 tablet_bootstrap.cc:654] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:11.414242 19237 tablet_bootstrap.cc:654] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:11.416059 19235 log.cc:824] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:11.416749 19236 tablet_bootstrap.cc:654] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:11.416749 19237 log.cc:824] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:11.419044 19236 log.cc:824] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:11.421854 19235 tablet_bootstrap.cc:492] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: No bootstrap required, opened a new log
I20260811 12:16:11.422004 19237 tablet_bootstrap.cc:492] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: No bootstrap required, opened a new log
I20260811 12:16:11.422268 19235 ts_tablet_manager.cc:1397] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260811 12:16:11.422426 19237 ts_tablet_manager.cc:1397] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.003s
I20260811 12:16:11.423810 19236 tablet_bootstrap.cc:492] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: No bootstrap required, opened a new log
I20260811 12:16:11.424193 19236 ts_tablet_manager.cc:1397] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.009s
I20260811 12:16:11.434633 19235 raft_consensus.cc:348] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.435307 19235 raft_consensus.cc:374] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:11.435559 19235 raft_consensus.cc:729] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9821077cf82e4c0c941244fbee6abf1e, State: Initialized, Role: FOLLOWER
I20260811 12:16:11.436400 19235 consensus_queue.cc:260] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.438796 19235 ts_tablet_manager.cc:1428] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: Time spent starting tablet: real 0.016s user 0.009s sys 0.007s
I20260811 12:16:11.438463 19236 raft_consensus.cc:348] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.439163 19236 raft_consensus.cc:374] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:11.439293 19236 raft_consensus.cc:729] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4039332f2847d9b9743590f94bafe1, State: Initialized, Role: FOLLOWER
I20260811 12:16:11.440864 19236 consensus_queue.cc:260] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.438279 19237 raft_consensus.cc:348] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.442962 19237 raft_consensus.cc:374] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:11.443332 19237 raft_consensus.cc:729] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a2cc2bb2de3418daf003e6958c41172, State: Initialized, Role: FOLLOWER
I20260811 12:16:11.444447 19237 consensus_queue.cc:260] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.445343 19236 ts_tablet_manager.cc:1428] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Time spent starting tablet: real 0.021s user 0.017s sys 0.006s
I20260811 12:16:11.445879 18967 heartbeater.cc:503] Master 127.7.39.62:37677 was elected leader, sending a full tablet report...
I20260811 12:16:11.448120 19186 heartbeater.cc:503] Master 127.7.39.62:37677 was elected leader, sending a full tablet report...
I20260811 12:16:11.449069 19237 ts_tablet_manager.cc:1428] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Time spent starting tablet: real 0.026s user 0.019s sys 0.003s
W20260811 12:16:11.472999 19189 tablet.cc:2369] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:16:11.497053 19242 raft_consensus.cc:482] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:11.497416 19242 raft_consensus.cc:504] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.499976 19242 leader_election.cc:288] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173), 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:11.504463 19245 raft_consensus.cc:482] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:11.504928 19245 raft_consensus.cc:504] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.511516 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.500482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:37929 (local address 127.7.39.3:42173)
0811 12:16:11.500796 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.500803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.501933 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 12:16:11.502171 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.502177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.502257 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.502359 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:11.503530 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.505047 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.508508 (+ 3461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.510216 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.510303 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.510413 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.511079 (+ 666us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.511229 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:16:11.512516 19248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.500162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:42173 (local address 127.7.39.1:37929)
0811 12:16:11.501807 (+ 1645us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.501833 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.502022 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.502389 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.502394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.502491 (+ 97us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:11.503368 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.503383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.505264 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.505270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.508323 (+ 3053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.508341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.510697 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.510705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.510885 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.510922 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.511722 (+ 800us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.511871 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1452,"mutex_wait_us":23,"spinlock_wait_cycles":5760,"thread_start_us":993,"threads_started":1}
I20260811 12:16:11.513113 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.500309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:35613 (local address 127.7.39.2:37651)
0811 12:16:11.500576 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.500583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.502027 (+ 1444us) server_negotiation.cc:407] Connection header received
0811 12:16:11.502182 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.502187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.502257 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.502359 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:11.503563 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.505056 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.508941 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.510690 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.510810 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.510933 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.512269 (+ 1336us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.512434 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:16:11.513223 19245 leader_election.cc:288] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645), 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:11.515027 19108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672dfb00d205412c803a357ca601eec8" candidate_uuid: "9821077cf82e4c0c941244fbee6abf1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a2cc2bb2de3418daf003e6958c41172" is_pre_election: true
I20260811 12:16:11.515662 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.500188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37651 (local address 127.7.39.1:35613)
0811 12:16:11.501922 (+ 1734us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.501946 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.502051 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.502389 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.502394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.502438 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:11.503366 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.503384 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.505264 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.505270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.508721 (+ 3451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.508739 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.511520 (+ 2781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.511528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.512037 (+ 509us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.512083 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.514354 (+ 2271us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.514624 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1507,"mutex_wait_us":103,"spinlock_wait_cycles":1408,"thread_start_us":980,"threads_started":1}
I20260811 12:16:11.515579 19108 raft_consensus.cc:2454] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9821077cf82e4c0c941244fbee6abf1e in term 0.
I20260811 12:16:11.516568 18537 leader_election.cc:302] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9821077cf82e4c0c941244fbee6abf1e, 9a2cc2bb2de3418daf003e6958c41172; no voters:
I20260811 12:16:11.517117 19242 raft_consensus.cc:2781] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:11.517419 19242 raft_consensus.cc:482] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:11.517666 19242 raft_consensus.cc:3037] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:11.518042 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.508988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:55847 (local address 127.7.39.1:35645)
0811 12:16:11.509250 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.509255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.509275 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:16:11.509334 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.509339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.509408 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.509497 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:11.510235 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.511419 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.514277 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.515848 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.515989 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.516080 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.517565 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.517725 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:16:11.517686 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672dfb00d205412c803a357ca601eec8" candidate_uuid: "9821077cf82e4c0c941244fbee6abf1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4039332f2847d9b9743590f94bafe1" is_pre_election: true
I20260811 12:16:11.518556 18886 raft_consensus.cc:2454] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9821077cf82e4c0c941244fbee6abf1e in term 0.
I20260811 12:16:11.519605 19193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.508189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:35645 (local address 127.7.39.3:55847)
0811 12:16:11.508600 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.508633 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.508734 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.509661 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.509666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.509693 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:11.510051 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.510063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.511594 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.511599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.513798 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.513816 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.516208 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.516214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.516641 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.516677 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.516788 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.519268 (+ 2480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":2300}
I20260811 12:16:11.520900 18649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672dfb00d205412c803a357ca601eec8" candidate_uuid: "9a2cc2bb2de3418daf003e6958c41172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9821077cf82e4c0c941244fbee6abf1e" is_pre_election: true
I20260811 12:16:11.521584 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.508246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37651 (local address 127.7.39.3:48879)
0811 12:16:11.508655 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:11.508674 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:11.508780 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:11.514016 (+ 5236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:11.514021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:11.514055 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:11.514385 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.514399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.516280 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.516285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.519091 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:11.519108 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.520776 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.520783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:11.521002 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.521039 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:16:11.521204 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 12:16:11.521355 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260811 12:16:11.521889 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.508986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:48879 (local address 127.7.39.2:37651)
0811 12:16:11.513661 (+ 4675us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.513667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.513686 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:16:11.513751 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.513790 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.513876 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.513983 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:11.514591 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.516029 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.519296 (+ 3267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.520596 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.520728 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.520832 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.521215 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.521361 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4557,"thread_start_us":4877,"threads_started":1}
I20260811 12:16:11.522431 19242 raft_consensus.cc:504] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.522589 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672dfb00d205412c803a357ca601eec8" candidate_uuid: "9a2cc2bb2de3418daf003e6958c41172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4039332f2847d9b9743590f94bafe1" is_pre_election: true
I20260811 12:16:11.522799 18886 raft_consensus.cc:2454] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a2cc2bb2de3418daf003e6958c41172 in term 0.
I20260811 12:16:11.523134 18649 raft_consensus.cc:2379] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a2cc2bb2de3418daf003e6958c41172 in current term 1: Already voted for candidate 9821077cf82e4c0c941244fbee6abf1e in this term.
I20260811 12:16:11.523316 19015 leader_election.cc:302] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4039332f2847d9b9743590f94bafe1, 9a2cc2bb2de3418daf003e6958c41172; no voters:
I20260811 12:16:11.523818 19245 raft_consensus.cc:2781] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:11.523954 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672dfb00d205412c803a357ca601eec8" candidate_uuid: "9821077cf82e4c0c941244fbee6abf1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4039332f2847d9b9743590f94bafe1"
I20260811 12:16:11.524087 19245 raft_consensus.cc:482] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:11.524190 18886 raft_consensus.cc:3037] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:11.524238 19245 raft_consensus.cc:3037] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:11.525143 19108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672dfb00d205412c803a357ca601eec8" candidate_uuid: "9821077cf82e4c0c941244fbee6abf1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a2cc2bb2de3418daf003e6958c41172"
I20260811 12:16:11.528478 18886 raft_consensus.cc:2454] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9821077cf82e4c0c941244fbee6abf1e in term 1.
I20260811 12:16:11.528496 19245 raft_consensus.cc:504] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.528751 19242 leader_election.cc:288] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [CANDIDATE]: Term 1 election: Requested vote from peers 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173), 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:11.529076 19108 raft_consensus.cc:2379] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9821077cf82e4c0c941244fbee6abf1e in current term 1: Already voted for candidate 9a2cc2bb2de3418daf003e6958c41172 in this term.
I20260811 12:16:11.529125 18538 leader_election.cc:302] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4039332f2847d9b9743590f94bafe1, 9821077cf82e4c0c941244fbee6abf1e; no voters:
I20260811 12:16:11.529771 19242 raft_consensus.cc:2781] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:11.530613 19242 raft_consensus.cc:686] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 1 LEADER]: Becoming Leader. State: Replica: 9821077cf82e4c0c941244fbee6abf1e, State: Running, Role: LEADER
I20260811 12:16:11.531519 19242 consensus_queue.cc:237] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:11.532006 18649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672dfb00d205412c803a357ca601eec8" candidate_uuid: "9a2cc2bb2de3418daf003e6958c41172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9821077cf82e4c0c941244fbee6abf1e"
I20260811 12:16:11.532487 19245 leader_election.cc:288] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [CANDIDATE]: Term 1 election: Requested vote from peers 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645), 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:11.532613 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672dfb00d205412c803a357ca601eec8" candidate_uuid: "9a2cc2bb2de3418daf003e6958c41172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4039332f2847d9b9743590f94bafe1"
I20260811 12:16:11.532953 18886 raft_consensus.cc:2379] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9a2cc2bb2de3418daf003e6958c41172 in current term 1: Already voted for candidate 9821077cf82e4c0c941244fbee6abf1e in this term.
I20260811 12:16:11.533747 19015 leader_election.cc:302] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a2cc2bb2de3418daf003e6958c41172; no voters: 0c4039332f2847d9b9743590f94bafe1, 9821077cf82e4c0c941244fbee6abf1e
I20260811 12:16:11.534137 19245 raft_consensus.cc:2726] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 12:16:11.537849 17134 catalog_manager.cc:5495] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1). New cstate: current_term: 1 leader_uuid: "9821077cf82e4c0c941244fbee6abf1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } health_report { overall_health: UNKNOWN } } }
W20260811 12:16:11.593448 18969 tablet.cc:2369] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:16:11.604992 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.598653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.7.39.1:35645)
0811 12:16:11.598810 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.598815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.598834 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:16:11.599047 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.599051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.599165 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.599301 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:11.599906 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.600859 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.602752 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.603216 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.603321 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.603875 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:11.604405 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:11.604569 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.604669 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.604750 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:16:11.616010 19108 raft_consensus.cc:1261] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Refusing update from remote peer 9821077cf82e4c0c941244fbee6abf1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:11.616057 18886 raft_consensus.cc:1261] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Refusing update from remote peer 9821077cf82e4c0c941244fbee6abf1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:11.616995 19251 consensus_queue.cc:1035] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:11.617635 19242 consensus_queue.cc:1035] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:11.637269 19256 mvcc.cc:205] Tried to move back new op lower bound from 7317301541326323712 to 7317301540998922240. Current Snapshot: MvccSnapshot[applied={T|T < 7317301541326323712}]
I20260811 12:16:11.668643 17134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:11.668942 17134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:11.677074 17134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 136c6fa878384eb88036b9fa7c3377b2: Sending DeleteTablet for 3 replicas of tablet 672dfb00d205412c803a357ca601eec8
I20260811 12:16:11.679358 19080 tablet_service.cc:1514] Processing DeleteTablet for tablet 672dfb00d205412c803a357ca601eec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:11 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:47220
I20260811 12:16:11.679509 18616 tablet_service.cc:1514] Processing DeleteTablet for tablet 672dfb00d205412c803a357ca601eec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:11 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:37348
I20260811 12:16:11.680698 19270 tablet_replica.cc:330] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: stopping tablet replica
I20260811 12:16:11.681214 18860 tablet_service.cc:1514] Processing DeleteTablet for tablet 672dfb00d205412c803a357ca601eec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:11 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:36720
I20260811 12:16:11.681385 19270 raft_consensus.cc:2229] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:11.683751 19270 raft_consensus.cc:2258] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:11.686407 19274 tablet_replica.cc:330] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: stopping tablet replica
I20260811 12:16:11.686954 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 17088
I20260811 12:16:11.686950 19270 ts_tablet_manager.cc:1905] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:11.686970 19274 raft_consensus.cc:2229] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:16:11.688083 17186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:11.688160 19274 raft_consensus.cc:2258] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:11.690373 19274 ts_tablet_manager.cc:1905] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:11.698706 19270 ts_tablet_manager.cc:1918] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:11.699113 19270 log.cc:1196] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/672dfb00d205412c803a357ca601eec8
I20260811 12:16:11.699383 19274 ts_tablet_manager.cc:1918] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:11.699630 19274 log.cc:1196] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/672dfb00d205412c803a357ca601eec8
I20260811 12:16:11.700109 19270 ts_tablet_manager.cc:1939] T 672dfb00d205412c803a357ca601eec8 P 9a2cc2bb2de3418daf003e6958c41172: Deleting consensus metadata
I20260811 12:16:11.700345 19274 ts_tablet_manager.cc:1939] T 672dfb00d205412c803a357ca601eec8 P 9821077cf82e4c0c941244fbee6abf1e: Deleting consensus metadata
I20260811 12:16:11.708340 19275 tablet_replica.cc:330] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: stopping tablet replica
I20260811 12:16:11.709404 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.682988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.7.39.61:39781)
0811 12:16:11.683661 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.683668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.685494 (+ 1826us) server_negotiation.cc:407] Connection header received
0811 12:16:11.686017 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.686022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.686112 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.686193 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.690385 (+ 4192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.691643 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.694271 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.695330 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.695447 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.700319 (+ 4872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.700328 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.700334 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.700386 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.704997 (+ 4611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.706377 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.706384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.706387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.706691 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.707318 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.707324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.707328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.707898 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.708259 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.708708 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.708965 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513}
I20260811 12:16:11.710464 19275 raft_consensus.cc:2229] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:11.711673 19275 raft_consensus.cc:2258] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:11.714402 19275 ts_tablet_manager.cc:1905] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:11.724859 19275 ts_tablet_manager.cc:1918] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:11.725246 19275 log.cc:1196] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/672dfb00d205412c803a357ca601eec8
I20260811 12:16:11.726186 19275 ts_tablet_manager.cc:1939] T 672dfb00d205412c803a357ca601eec8 P 0c4039332f2847d9b9743590f94bafe1: Deleting consensus metadata
I20260811 12:16:11.732892 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.685706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.7.39.60:46679)
0811 12:16:11.685898 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.685905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.709575 (+ 23670us) server_negotiation.cc:407] Connection header received
0811 12:16:11.709889 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.709895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.709984 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.710062 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.713492 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.714917 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.717770 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.718684 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.718792 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.724339 (+ 5547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.724347 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.724353 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.724403 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.728991 (+ 4588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.730145 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.730152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.730155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.730349 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.731010 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.731016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.731019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.731474 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.731746 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.732208 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.732472 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:16:11.840197 17118 catalog_manager.cc:4841] TS 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651): tablet 672dfb00d205412c803a357ca601eec8 (table test-table [id=a9ea43ad47e74d59b205daf4c65f6ea2]) successfully deleted
I20260811 12:16:11.845846 17117 catalog_manager.cc:4841] TS 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645): tablet 672dfb00d205412c803a357ca601eec8 (table test-table [id=a9ea43ad47e74d59b205daf4c65f6ea2]) successfully deleted
W20260811 12:16:11.846129 17117 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.39.1:35645, user_credentials={real_user=slave}} blocked reactor thread for 143743us
I20260811 12:16:11.846608 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 17088
I20260811 12:16:11.846644 17117 catalog_manager.cc:4841] TS 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173): tablet 672dfb00d205412c803a357ca601eec8 (table test-table [id=a9ea43ad47e74d59b205daf4c65f6ea2]) successfully deleted
I20260811 12:16:11.923866 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.889383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.7.39.61:39781)
0811 12:16:11.889617 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.889624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.901708 (+ 12084us) server_negotiation.cc:407] Connection header received
0811 12:16:11.903286 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.903293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.903386 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.903463 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.908165 (+ 4702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.909362 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.911878 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.912632 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.912735 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.916537 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.916544 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.916549 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.916592 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.920415 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.921392 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.921398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.921402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.921560 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.922026 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.922031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.922035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.922412 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.922656 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.923259 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.923632 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:16:11.923915 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:11.889767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.7.39.60:46679)
0811 12:16:11.890000 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:11.890008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:11.904463 (+ 14455us) server_negotiation.cc:407] Connection header received
0811 12:16:11.904987 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:11.904994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:11.905085 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:11.905193 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:11.907680 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.908872 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.911196 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:11.911868 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:11.911968 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:11.916522 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:11.916540 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:11.916545 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:11.916589 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:11.920304 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.921384 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.921390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.921394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.921550 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:11.921972 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:11.921976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:11.921979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:11.922357 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:11.922595 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:11.923244 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 12:16:11.923493 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:16:12.650863 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:12.647198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:49097)
0811 12:16:12.649526 (+ 2328us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:12.650105 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1606,"thread_start_us":1120,"threads_started":1}
W20260811 12:16:12.652686 18721 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:37677 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:12.659511 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:12.657851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:49851)
0811 12:16:12.659134 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:12.659295 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":545,"threads_started":1}
W20260811 12:16:12.660347 18967 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:37677 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:12.664707 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:12.662261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:38477)
0811 12:16:12.663631 (+ 1370us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:12.664187 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1072,"thread_start_us":792,"threads_started":1}
W20260811 12:16:12.665704 19186 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:37677 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:13.178872 19291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 136c6fa878384eb88036b9fa7c3377b2)
I20260811 12:16:13.179586 19292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 136c6fa878384eb88036b9fa7c3377b2)
I20260811 12:16:13.179401 19291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } }
I20260811 12:16:13.180042 19292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } }
I20260811 12:16:13.182717 19291 leader_election.cc:288] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677), 1b5602e8998c40879c210fb36631f753 (127.7.39.61:39781)
I20260811 12:16:13.182976 19292 leader_election.cc:288] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677), 68ee124b36cd40aa9df52c75d2662f24 (127.7.39.60:46679)
I20260811 12:16:13.198810 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.183494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34434)
0811 12:16:13.197779 (+ 14285us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.198187 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14072,"thread_start_us":9661,"threads_started":1}
I20260811 12:16:13.199523 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.182780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34432)
0811 12:16:13.198599 (+ 15819us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.199022 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15596,"thread_start_us":15287,"threads_started":1}
W20260811 12:16:13.199579 18106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
W20260811 12:16:13.200419 18389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:13.212999 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.212183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34448)
0811 12:16:13.212500 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.212654 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:16:13.213698 18389 leader_election.cc:334] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677): Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:13.213888 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.212845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34456)
0811 12:16:13.213408 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.213619 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260811 12:16:13.214439 18106 leader_election.cc:334] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677): Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:13.216743 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.198689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.7.39.60:46679)
0811 12:16:13.200354 (+ 1665us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:13.200367 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:13.200412 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:16:13.200518 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:13.200522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:13.200643 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:13.202434 (+ 1791us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:13.202886 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:13.205626 (+ 2740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:13.210748 (+ 5122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:13.214192 (+ 3444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:13.215494 (+ 1302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:13.215755 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:13.215891 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 12:16:13.216158 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":1455,"thread_start_us":1192,"threads_started":1}
I20260811 12:16:13.216764 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.183455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:46679 (local address 127.0.0.1:40590)
0811 12:16:13.193840 (+ 10385us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.193893 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:13.194205 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:13.201088 (+ 6883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:13.201094 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:13.201134 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:13.202095 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:13.202122 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:13.205985 (+ 3863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:13.206004 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:13.210265 (+ 4261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:13.210290 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:13.214689 (+ 4399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:13.214707 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:13.215408 (+ 701us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:13.215484 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 12:16:13.215610 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 12:16:13.215956 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10044,"mutex_wait_us":1,"thread_start_us":13906,"threads_started":1}
I20260811 12:16:13.217481 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.183904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.7.39.61:39781)
0811 12:16:13.196858 (+ 12954us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:13.196869 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:13.199197 (+ 2328us) server_negotiation.cc:407] Connection header received
0811 12:16:13.199656 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:13.199663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:13.199829 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:13.199938 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:13.201461 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:13.204032 (+ 2571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:13.210149 (+ 6117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:13.212526 (+ 2377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:13.212751 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:13.212938 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:13.215851 (+ 2913us) server_negotiation.cc:299] Negotiation successful
0811 12:16:13.216150 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12624,"thread_start_us":12953,"threads_started":1}
I20260811 12:16:13.218333 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.182967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:39781 (local address 127.0.0.1:41158)
0811 12:16:13.198999 (+ 16032us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.199026 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:13.199357 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:13.200359 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:13.200374 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:13.200433 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:13.200977 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:13.200994 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:13.204525 (+ 3531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:13.204542 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:13.208954 (+ 4412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:13.209001 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:13.212832 (+ 3831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:13.212842 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:13.215353 (+ 2511us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:13.215553 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 12:16:13.217365 (+ 1812us) client_negotiation.cc:240] Negotiation successful
0811 12:16:13.217598 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15817,"mutex_wait_us":86,"thread_start_us":15348,"threads_started":1}
I20260811 12:16:13.218114 18435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b5602e8998c40879c210fb36631f753" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68ee124b36cd40aa9df52c75d2662f24" is_pre_election: true
I20260811 12:16:13.218880 18435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b5602e8998c40879c210fb36631f753 in term 1.
I20260811 12:16:13.220397 18110 leader_election.cc:302] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b5602e8998c40879c210fb36631f753, 68ee124b36cd40aa9df52c75d2662f24; no voters: 136c6fa878384eb88036b9fa7c3377b2
I20260811 12:16:13.220582 18145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ee124b36cd40aa9df52c75d2662f24" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1b5602e8998c40879c210fb36631f753" is_pre_election: true
I20260811 12:16:13.221280 18145 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ee124b36cd40aa9df52c75d2662f24 in term 1.
I20260811 12:16:13.221539 19292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 12:16:13.221697 19292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 136c6fa878384eb88036b9fa7c3377b2)
I20260811 12:16:13.221781 19292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:16:13.222045 18392 leader_election.cc:302] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b5602e8998c40879c210fb36631f753, 68ee124b36cd40aa9df52c75d2662f24; no voters: 136c6fa878384eb88036b9fa7c3377b2
I20260811 12:16:13.222607 19291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 12:16:13.222867 19291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 136c6fa878384eb88036b9fa7c3377b2)
I20260811 12:16:13.223037 19291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:16:13.225481 19292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } }
I20260811 12:16:13.226593 19292 leader_election.cc:288] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [CANDIDATE]: Term 2 election: Requested vote from peers 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677), 68ee124b36cd40aa9df52c75d2662f24 (127.7.39.60:46679)
I20260811 12:16:13.227725 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.226856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34460)
0811 12:16:13.227364 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.227516 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257}
I20260811 12:16:13.227543 19291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } }
I20260811 12:16:13.227828 18435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b5602e8998c40879c210fb36631f753" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68ee124b36cd40aa9df52c75d2662f24"
I20260811 12:16:13.228448 18435 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b5602e8998c40879c210fb36631f753 in current term 2: Already voted for candidate 68ee124b36cd40aa9df52c75d2662f24 in this term.
I20260811 12:16:13.229001 19291 leader_election.cc:288] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 2 election: Requested vote from peers 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677), 1b5602e8998c40879c210fb36631f753 (127.7.39.61:39781)
I20260811 12:16:13.229700 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.228931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34472)
0811 12:16:13.229263 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.229445 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 12:16:13.229743 18145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ee124b36cd40aa9df52c75d2662f24" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1b5602e8998c40879c210fb36631f753"
W20260811 12:16:13.230095 18106 leader_election.cc:334] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677): Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:13.230350 18145 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 68ee124b36cd40aa9df52c75d2662f24 in current term 2: Already voted for candidate 1b5602e8998c40879c210fb36631f753 in this term.
I20260811 12:16:13.231302 18110 leader_election.cc:302] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b5602e8998c40879c210fb36631f753; no voters: 136c6fa878384eb88036b9fa7c3377b2, 68ee124b36cd40aa9df52c75d2662f24
I20260811 12:16:13.231468 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.229099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34486)
0811 12:16:13.230994 (+ 1895us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.231211 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 12:16:13.231935 19292 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 12:16:13.233753 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.233018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34516)
0811 12:16:13.233296 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.233481 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 12:16:13.234308 18389 leader_election.cc:334] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677): Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:13.234515 18389 leader_election.cc:302] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68ee124b36cd40aa9df52c75d2662f24; no voters: 136c6fa878384eb88036b9fa7c3377b2, 1b5602e8998c40879c210fb36631f753
I20260811 12:16:13.235023 19291 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 12:16:13.654405 19306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.653500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:56257)
0811 12:16:13.654022 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.654120 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":288,"threads_started":1}
I20260811 12:16:13.661944 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.661134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:39693)
0811 12:16:13.661700 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.661785 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":326,"threads_started":1}
I20260811 12:16:13.667166 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:13.666538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:60499)
0811 12:16:13.666918 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:13.666996 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":198,"threads_started":1}
I20260811 12:16:14.656239 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.655350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:45225)
0811 12:16:14.655910 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.656020 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":291,"threads_started":1}
I20260811 12:16:14.663587 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.662789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:55679)
0811 12:16:14.663310 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.663400 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":257,"threads_started":1}
I20260811 12:16:14.668653 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.667957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:52241)
0811 12:16:14.668386 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.668475 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":219,"threads_started":1}
I20260811 12:16:14.853130 19317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:14.853402 19317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } }
I20260811 12:16:14.854079 19317 leader_election.cc:288] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677), 1b5602e8998c40879c210fb36631f753 (127.7.39.61:39781)
I20260811 12:16:14.854681 18145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ee124b36cd40aa9df52c75d2662f24" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1b5602e8998c40879c210fb36631f753" is_pre_election: true
I20260811 12:16:14.854930 18145 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68ee124b36cd40aa9df52c75d2662f24 in term 2.
I20260811 12:16:14.855067 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.854227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34748)
0811 12:16:14.854741 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.854842 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":260,"threads_started":1}
I20260811 12:16:14.855469 18392 leader_election.cc:302] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b5602e8998c40879c210fb36631f753, 68ee124b36cd40aa9df52c75d2662f24; no voters:
I20260811 12:16:14.855800 19317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 12:16:14.855979 19317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:14.856137 19317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:16:14.856673 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.856291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34764)
0811 12:16:14.856444 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.856529 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 12:16:14.856869 18389 leader_election.cc:334] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677): Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:14.858487 19317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } }
I20260811 12:16:14.859052 19317 leader_election.cc:288] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 3 election: Requested vote from peers 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677), 1b5602e8998c40879c210fb36631f753 (127.7.39.61:39781)
I20260811 12:16:14.859625 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.859262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34776)
0811 12:16:14.859416 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.859486 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:16:14.859597 18145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ee124b36cd40aa9df52c75d2662f24" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1b5602e8998c40879c210fb36631f753"
I20260811 12:16:14.859818 18145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:16:14.860548 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.860227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34780)
0811 12:16:14.860376 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.860442 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:16:14.860711 18389 leader_election.cc:334] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677): Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
I20260811 12:16:14.862205 18145 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68ee124b36cd40aa9df52c75d2662f24 in term 3.
I20260811 12:16:14.862676 18392 leader_election.cc:302] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b5602e8998c40879c210fb36631f753, 68ee124b36cd40aa9df52c75d2662f24; no voters: 136c6fa878384eb88036b9fa7c3377b2
I20260811 12:16:14.863037 19317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:16:14.863469 19317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [term 3 LEADER]: Becoming Leader. State: Replica: 68ee124b36cd40aa9df52c75d2662f24, State: Running, Role: LEADER
I20260811 12:16:14.863873 19317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } }
I20260811 12:16:14.866052 19322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68ee124b36cd40aa9df52c75d2662f24. Latest consensus state: current_term: 3 leader_uuid: "68ee124b36cd40aa9df52c75d2662f24" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:14.866386 19322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:14.867041 19326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:14.871162 19326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:14.872030 19326 catalog_manager.cc:1259] Loaded cluster ID: 7a252f3444ae484aa05fd7c3110b1bfe
I20260811 12:16:14.872099 19326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:14.873350 19326 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:14.874145 19326 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Loaded TSK: 0
I20260811 12:16:14.875068 19326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:16:14.890995 18418 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:16:14.898278 18418 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:16:14.902911 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.902467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34800)
0811 12:16:14.902674 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.902760 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 12:16:14.902971 18145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [term 3 FOLLOWER]: Refusing update from remote peer 68ee124b36cd40aa9df52c75d2662f24: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 12:16:14.903822 19322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 12:16:14.904022 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.903619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34804)
0811 12:16:14.903814 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.903885 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:16:14.904389 18389 consensus_peers.cc:487] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 -> Peer 136c6fa878384eb88036b9fa7c3377b2 (127.7.39.62:37677): Couldn't send request to peer 136c6fa878384eb88036b9fa7c3377b2. Status: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 12:16:14.907131 19328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68ee124b36cd40aa9df52c75d2662f24. Latest consensus state: current_term: 3 leader_uuid: "68ee124b36cd40aa9df52c75d2662f24" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:14.907487 19328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:16:14.908824 19317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "68ee124b36cd40aa9df52c75d2662f24" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:14.909181 19317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:14.909309 19328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "68ee124b36cd40aa9df52c75d2662f24" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "136c6fa878384eb88036b9fa7c3377b2" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 37677 } } peers { permanent_uuid: "1b5602e8998c40879c210fb36631f753" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "68ee124b36cd40aa9df52c75d2662f24" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 46679 } attrs { promote: false } } }
I20260811 12:16:14.909564 19328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5602e8998c40879c210fb36631f753 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:16:14.926874 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.918212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:42173 (local address 127.0.0.1:47226)
0811 12:16:14.918705 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.918724 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:14.918850 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:14.919649 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:14.919655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:14.919686 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:14.920097 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:14.920111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.921499 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.921505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:14.923661 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:14.923675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.925570 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.925577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:14.926230 (+ 653us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:14.926270 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:16:14.926346 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 12:16:14.926496 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":430}
I20260811 12:16:14.927914 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.918404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.7.39.3:42173)
0811 12:16:14.919296 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:14.919302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:14.919322 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:16:14.919396 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:14.919401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:14.919487 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:14.919593 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:14.920240 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.921317 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.923844 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.925132 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.927271 (+ 2139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:14.927394 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:14.927448 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 12:16:14.927596 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":643,"threads_started":1}
I20260811 12:16:14.928851 19080 tablet_service.cc:1467] Processing CreateTablet for tablet 11ee1abb30354e45916bbd7dcabf14da (DEFAULT_TABLE table=test-table [id=03c12e59e74f40c98f048018bba6e520]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:14.929491 19080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ee1abb30354e45916bbd7dcabf14da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:14.929643 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.919727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.7.39.2:37651)
0811 12:16:14.921946 (+ 2219us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:14.921952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:14.922741 (+ 789us) server_negotiation.cc:407] Connection header received
0811 12:16:14.922820 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:14.922825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:14.922897 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:14.922989 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:14.923603 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.924672 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.927186 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.928356 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.928454 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:14.928566 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:14.929119 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 12:16:14.929288 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2082,"thread_start_us":1961,"threads_started":1}
I20260811 12:16:14.929791 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.918673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:35645 (local address 127.0.0.1:37372)
0811 12:16:14.920399 (+ 1726us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.920412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:14.920495 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:14.923991 (+ 3496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:14.923995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:14.924014 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:14.924315 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:14.924323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.925703 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.925708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:14.927507 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:14.927521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.928934 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.928941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:14.929100 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:14.929141 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:16:14.929256 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:16:14.929395 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"mutex_wait_us":2,"thread_start_us":1255,"threads_started":1}
I20260811 12:16:14.930552 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.919364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37651 (local address 127.0.0.1:36734)
0811 12:16:14.922426 (+ 3062us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.922445 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:14.922577 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:14.923048 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:14.923054 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:14.923085 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:14.923428 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:14.923440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.924854 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.924859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:14.926983 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:14.927000 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.928730 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.928735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:14.928919 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:14.928951 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:16:14.930028 (+ 1077us) client_negotiation.cc:240] Negotiation successful
0811 12:16:14.930166 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2949,"mutex_wait_us":1,"thread_start_us":2756,"threads_started":1}
I20260811 12:16:14.930676 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.918927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.7.39.1:35645)
0811 12:16:14.923501 (+ 4574us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:14.923508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:14.923529 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:16:14.923602 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:14.923617 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:14.923707 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:14.923801 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:14.924511 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.925420 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.927684 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:14.928692 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:14.928820 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:14.928934 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:14.929317 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 12:16:14.929463 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4453,"thread_start_us":4325,"threads_started":1}
I20260811 12:16:14.931602 18616 tablet_service.cc:1467] Processing CreateTablet for tablet 11ee1abb30354e45916bbd7dcabf14da (DEFAULT_TABLE table=test-table [id=03c12e59e74f40c98f048018bba6e520]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:14.931982 18860 tablet_service.cc:1467] Processing CreateTablet for tablet 11ee1abb30354e45916bbd7dcabf14da (DEFAULT_TABLE table=test-table [id=03c12e59e74f40c98f048018bba6e520]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:14.932135 18616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ee1abb30354e45916bbd7dcabf14da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:14.932437 18860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ee1abb30354e45916bbd7dcabf14da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:14.935424 19343 tablet_bootstrap.cc:492] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: Bootstrap starting.
I20260811 12:16:14.937239 19343 tablet_bootstrap.cc:654] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:14.939266 19343 tablet_bootstrap.cc:492] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: No bootstrap required, opened a new log
I20260811 12:16:14.939435 19343 ts_tablet_manager.cc:1397] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 12:16:14.939829 19346 tablet_bootstrap.cc:492] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: Bootstrap starting.
I20260811 12:16:14.940479 19343 raft_consensus.cc:348] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:14.940749 19343 raft_consensus.cc:374] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:14.940853 19343 raft_consensus.cc:729] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a2cc2bb2de3418daf003e6958c41172, State: Initialized, Role: FOLLOWER
I20260811 12:16:14.941107 19343 consensus_queue.cc:260] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:14.941814 19348 tablet_bootstrap.cc:492] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: Bootstrap starting.
I20260811 12:16:14.942565 19346 tablet_bootstrap.cc:654] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:14.943601 19348 tablet_bootstrap.cc:654] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:14.945194 19346 tablet_bootstrap.cc:492] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: No bootstrap required, opened a new log
I20260811 12:16:14.945361 19346 ts_tablet_manager.cc:1397] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 12:16:14.945799 19348 tablet_bootstrap.cc:492] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: No bootstrap required, opened a new log
I20260811 12:16:14.945937 19348 ts_tablet_manager.cc:1397] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 12:16:14.946249 19346 raft_consensus.cc:348] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:14.946516 19346 raft_consensus.cc:374] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:14.946610 19346 raft_consensus.cc:729] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4039332f2847d9b9743590f94bafe1, State: Initialized, Role: FOLLOWER
I20260811 12:16:14.946630 19348 raft_consensus.cc:348] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:14.946871 19348 raft_consensus.cc:374] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:14.946970 19348 raft_consensus.cc:729] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9821077cf82e4c0c941244fbee6abf1e, State: Initialized, Role: FOLLOWER
I20260811 12:16:14.946877 19346 consensus_queue.cc:260] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:14.947228 19348 consensus_queue.cc:260] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:14.948035 19348 ts_tablet_manager.cc:1428] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:16:14.948249 19343 ts_tablet_manager.cc:1428] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260811 12:16:14.949117 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.948644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:37971)
0811 12:16:14.948827 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.948937 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 12:16:14.949795 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.949336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:56007)
0811 12:16:14.949523 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.949633 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:16:14.951242 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.950674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:53865)
0811 12:16:14.950932 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.951034 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 12:16:14.953531 19185 heartbeater.cc:503] Master 127.7.39.60:46679 was elected leader, sending a full tablet report...
I20260811 12:16:14.953822 18720 heartbeater.cc:503] Master 127.7.39.60:46679 was elected leader, sending a full tablet report...
I20260811 12:16:14.956142 19346 ts_tablet_manager.cc:1428] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20260811 12:16:14.956714 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.956198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:42173)
0811 12:16:14.956444 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.956552 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 12:16:14.957475 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.956821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:57387)
0811 12:16:14.957193 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.957296 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
I20260811 12:16:14.957962 18965 heartbeater.cc:503] Master 127.7.39.60:46679 was elected leader, sending a full tablet report...
I20260811 12:16:14.958485 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:14.957825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:43631)
0811 12:16:14.958252 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:14.958323 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342}
I20260811 12:16:15.135138 19353 raft_consensus.cc:482] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:15.135242 19349 raft_consensus.cc:482] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:15.135445 19353 raft_consensus.cc:504] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:15.135452 19349 raft_consensus.cc:504] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:15.136350 19353 leader_election.cc:288] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173), 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:15.136596 19349 leader_election.cc:288] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645), 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:15.136875 19108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee1abb30354e45916bbd7dcabf14da" candidate_uuid: "9821077cf82e4c0c941244fbee6abf1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a2cc2bb2de3418daf003e6958c41172" is_pre_election: true
I20260811 12:16:15.136997 18649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee1abb30354e45916bbd7dcabf14da" candidate_uuid: "9a2cc2bb2de3418daf003e6958c41172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9821077cf82e4c0c941244fbee6abf1e" is_pre_election: true
I20260811 12:16:15.137050 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee1abb30354e45916bbd7dcabf14da" candidate_uuid: "9a2cc2bb2de3418daf003e6958c41172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4039332f2847d9b9743590f94bafe1" is_pre_election: true
I20260811 12:16:15.137183 19108 raft_consensus.cc:2454] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9821077cf82e4c0c941244fbee6abf1e in term 0.
I20260811 12:16:15.137071 18885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee1abb30354e45916bbd7dcabf14da" candidate_uuid: "9821077cf82e4c0c941244fbee6abf1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4039332f2847d9b9743590f94bafe1" is_pre_election: true
I20260811 12:16:15.137306 18649 raft_consensus.cc:2454] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a2cc2bb2de3418daf003e6958c41172 in term 0.
I20260811 12:16:15.137372 18886 raft_consensus.cc:2454] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a2cc2bb2de3418daf003e6958c41172 in term 0.
I20260811 12:16:15.137770 19013 leader_election.cc:302] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9821077cf82e4c0c941244fbee6abf1e, 9a2cc2bb2de3418daf003e6958c41172; no voters:
I20260811 12:16:15.137929 18537 leader_election.cc:302] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9821077cf82e4c0c941244fbee6abf1e, 9a2cc2bb2de3418daf003e6958c41172; no voters: 0c4039332f2847d9b9743590f94bafe1
I20260811 12:16:15.138063 19349 raft_consensus.cc:2781] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:15.138178 19349 raft_consensus.cc:482] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:15.138240 19353 raft_consensus.cc:2781] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:15.138271 19349 raft_consensus.cc:3037] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:15.138358 19353 raft_consensus.cc:482] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:15.138453 19353 raft_consensus.cc:3037] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:15.140443 19349 raft_consensus.cc:504] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:15.140534 19353 raft_consensus.cc:504] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:15.141019 19349 leader_election.cc:288] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [CANDIDATE]: Term 1 election: Requested vote from peers 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645), 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:15.141094 19353 leader_election.cc:288] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [CANDIDATE]: Term 1 election: Requested vote from peers 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173), 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651)
I20260811 12:16:15.141644 18649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee1abb30354e45916bbd7dcabf14da" candidate_uuid: "9a2cc2bb2de3418daf003e6958c41172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9821077cf82e4c0c941244fbee6abf1e"
I20260811 12:16:15.141680 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee1abb30354e45916bbd7dcabf14da" candidate_uuid: "9821077cf82e4c0c941244fbee6abf1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4039332f2847d9b9743590f94bafe1"
I20260811 12:16:15.141680 18885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee1abb30354e45916bbd7dcabf14da" candidate_uuid: "9a2cc2bb2de3418daf003e6958c41172" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4039332f2847d9b9743590f94bafe1"
I20260811 12:16:15.141772 19108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ee1abb30354e45916bbd7dcabf14da" candidate_uuid: "9821077cf82e4c0c941244fbee6abf1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a2cc2bb2de3418daf003e6958c41172"
I20260811 12:16:15.141927 18649 raft_consensus.cc:2379] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9a2cc2bb2de3418daf003e6958c41172 in current term 1: Already voted for candidate 9821077cf82e4c0c941244fbee6abf1e in this term.
I20260811 12:16:15.141913 18886 raft_consensus.cc:3037] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:15.142027 19108 raft_consensus.cc:2379] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9821077cf82e4c0c941244fbee6abf1e in current term 1: Already voted for candidate 9a2cc2bb2de3418daf003e6958c41172 in this term.
I20260811 12:16:15.142458 19013 leader_election.cc:302] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a2cc2bb2de3418daf003e6958c41172; no voters: 0c4039332f2847d9b9743590f94bafe1, 9821077cf82e4c0c941244fbee6abf1e
I20260811 12:16:15.142764 19349 raft_consensus.cc:2726] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 12:16:15.144258 18886 raft_consensus.cc:2454] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9821077cf82e4c0c941244fbee6abf1e in term 1.
I20260811 12:16:15.144716 18538 leader_election.cc:302] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c4039332f2847d9b9743590f94bafe1, 9821077cf82e4c0c941244fbee6abf1e; no voters: 9a2cc2bb2de3418daf003e6958c41172
I20260811 12:16:15.144999 19353 raft_consensus.cc:2781] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:15.145491 19353 raft_consensus.cc:686] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 1 LEADER]: Becoming Leader. State: Replica: 9821077cf82e4c0c941244fbee6abf1e, State: Running, Role: LEADER
I20260811 12:16:15.145804 19353 consensus_queue.cc:237] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } }
I20260811 12:16:15.146574 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.146125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:44211)
0811 12:16:15.146292 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:15.146413 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:16:15.148869 18418 catalog_manager.cc:5495] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1). New cstate: current_term: 1 leader_uuid: "9821077cf82e4c0c941244fbee6abf1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9821077cf82e4c0c941244fbee6abf1e" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 } health_report { overall_health: UNKNOWN } } }
I20260811 12:16:15.174674 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.170067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.7.39.1:35645)
0811 12:16:15.170286 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:15.170291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:15.170311 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:16:15.170438 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:15.170442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:15.170511 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:15.170594 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:15.170913 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:15.171603 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:15.172936 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:15.173420 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:15.173542 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:15.173958 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:15.174086 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:15.174167 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:15.174317 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 12:16:15.174403 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:16:15.178285 19108 raft_consensus.cc:1261] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Refusing update from remote peer 9821077cf82e4c0c941244fbee6abf1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:15.178292 18886 raft_consensus.cc:1261] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Refusing update from remote peer 9821077cf82e4c0c941244fbee6abf1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:15.178980 19355 consensus_queue.cc:1035] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c4039332f2847d9b9743590f94bafe1" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:15.179278 19353 consensus_queue.cc:1035] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a2cc2bb2de3418daf003e6958c41172" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:15.182328 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.181781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:37745)
0811 12:16:15.182038 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:15.182148 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 12:16:15.182863 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.181818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:35041)
0811 12:16:15.182598 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:15.182694 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":666}
W20260811 12:16:15.183204 18723 tablet.cc:2369] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:16:15.188446 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.187998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.1:34463)
0811 12:16:15.188157 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:15.188258 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 12:16:15.189127 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.188587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.3:52317)
0811 12:16:15.188856 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:15.188939 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 12:16:15.193590 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.191060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.7.39.2:42943)
0811 12:16:15.193328 (+ 2268us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:15.193433 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2133}
I20260811 12:16:15.195528 18418 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:15.195760 18418 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:15.200966 18418 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68ee124b36cd40aa9df52c75d2662f24: Sending DeleteTablet for 3 replicas of tablet 11ee1abb30354e45916bbd7dcabf14da
I20260811 12:16:15.202020 19080 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ee1abb30354e45916bbd7dcabf14da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:15 UTC) from {username='oryx', principal='oryx/127.7.39.60@KRBTEST.COM'} at 127.0.0.1:47226
I20260811 12:16:15.202328 18860 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ee1abb30354e45916bbd7dcabf14da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:15 UTC) from {username='oryx', principal='oryx/127.7.39.60@KRBTEST.COM'} at 127.0.0.1:36734
I20260811 12:16:15.202476 18616 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ee1abb30354e45916bbd7dcabf14da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:15 UTC) from {username='oryx', principal='oryx/127.7.39.60@KRBTEST.COM'} at 127.0.0.1:37372
I20260811 12:16:15.203128 19371 tablet_replica.cc:330] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: stopping tablet replica
I20260811 12:16:15.203306 19371 raft_consensus.cc:2229] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:15.203577 19371 raft_consensus.cc:2258] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:15.204855 19372 tablet_replica.cc:330] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: stopping tablet replica
I20260811 12:16:15.205016 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 18363
I20260811 12:16:15.205060 19372 raft_consensus.cc:2229] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:16:15.205256 19371 ts_tablet_manager.cc:1905] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:15.205540 19372 raft_consensus.cc:2258] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:15.207165 19372 ts_tablet_manager.cc:1905] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:15.207249 19370 tablet_replica.cc:330] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: stopping tablet replica
I20260811 12:16:15.207446 19370 raft_consensus.cc:2229] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:15.207773 19370 raft_consensus.cc:2258] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:15.208938 19370 ts_tablet_manager.cc:1905] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:15.212143 19371 ts_tablet_manager.cc:1918] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:15.212320 19371 log.cc:1196] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/11ee1abb30354e45916bbd7dcabf14da
I20260811 12:16:15.212832 19371 ts_tablet_manager.cc:1939] T 11ee1abb30354e45916bbd7dcabf14da P 0c4039332f2847d9b9743590f94bafe1: Deleting consensus metadata
I20260811 12:16:15.214247 18393 catalog_manager.cc:4841] TS 0c4039332f2847d9b9743590f94bafe1 (127.7.39.2:37651): tablet 11ee1abb30354e45916bbd7dcabf14da (table test-table [id=03c12e59e74f40c98f048018bba6e520]) successfully deleted
I20260811 12:16:15.215459 19372 ts_tablet_manager.cc:1918] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:15.215620 19372 log.cc:1196] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/11ee1abb30354e45916bbd7dcabf14da
I20260811 12:16:15.215903 19370 ts_tablet_manager.cc:1918] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:15.216054 19370 log.cc:1196] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/11ee1abb30354e45916bbd7dcabf14da
I20260811 12:16:15.216137 19372 ts_tablet_manager.cc:1939] T 11ee1abb30354e45916bbd7dcabf14da P 9821077cf82e4c0c941244fbee6abf1e: Deleting consensus metadata
I20260811 12:16:15.216460 19370 ts_tablet_manager.cc:1939] T 11ee1abb30354e45916bbd7dcabf14da P 9a2cc2bb2de3418daf003e6958c41172: Deleting consensus metadata
I20260811 12:16:15.217396 18392 catalog_manager.cc:4841] TS 9821077cf82e4c0c941244fbee6abf1e (127.7.39.1:35645): tablet 11ee1abb30354e45916bbd7dcabf14da (table test-table [id=03c12e59e74f40c98f048018bba6e520]) successfully deleted
I20260811 12:16:15.217710 18392 catalog_manager.cc:4841] TS 9a2cc2bb2de3418daf003e6958c41172 (127.7.39.3:42173): tablet 11ee1abb30354e45916bbd7dcabf14da (table test-table [id=03c12e59e74f40c98f048018bba6e520]) successfully deleted
I20260811 12:16:15.226722 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.206333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.7.39.1:35645)
0811 12:16:15.206530 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:15.206536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:15.207142 (+ 606us) server_negotiation.cc:407] Connection header received
0811 12:16:15.207454 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:15.207460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:15.207541 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:15.207618 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:15.209633 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:15.210814 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:15.212649 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:15.213395 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:15.213514 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:15.220340 (+ 6826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:15.220359 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:15.220378 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:15.220423 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:15.223578 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:15.224371 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:15.224380 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:15.224383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:15.224556 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:15.224955 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:15.224960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:15.224962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:15.225470 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:15.225785 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:15.226107 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 12:16:15.226330 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:16:15.227489 18710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:15.351735 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 18363
I20260811 12:16:15.381563 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 18748
I20260811 12:16:15.395244 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.382395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36738 (local address 127.7.39.2:37651)
0811 12:16:15.382573 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:15.382577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:15.382597 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:16:15.382768 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:15.382771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:15.382848 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:15.382934 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:15.384209 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:15.384945 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:15.386414 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:15.386888 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:15.387024 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:15.389657 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:15.389701 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:15.389719 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:15.389762 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:15.392551 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:15.393362 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:15.393371 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:15.393373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:15.393520 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:15.393866 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:15.393869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:15.393871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:15.394269 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:15.394486 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:15.394794 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 12:16:15.394980 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:16:15.395968 18957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:15.411093 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.410537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34818)
0811 12:16:15.410748 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:15.410894 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 12:16:15.412488 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.412072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:37677 (local address 127.0.0.1:34830)
0811 12:16:15.412250 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:15.412342 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:37677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:16:15.494802 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 18748
I20260811 12:16:15.521574 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 18984
I20260811 12:16:15.536087 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:15.522448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.7.39.3:42173)
0811 12:16:15.522910 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:15.522915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:15.522933 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:16:15.522994 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:15.522997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:15.523060 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:15.523161 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:15.524548 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:15.525454 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:15.526842 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:15.527294 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:15.527412 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:15.530137 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:15.530157 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:15.530177 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:15.530218 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:15.533217 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:15.533975 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:15.533984 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:15.533987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:15.534150 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:15.534592 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:15.534596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:15.534599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:15.535006 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:15.535252 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:15.535515 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 12:16:15.535739 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":245,"threads_started":1}
I20260811 12:16:15.536859 19176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:15.632715 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 18984
I20260811 12:16:15.659922 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 17914
I20260811 12:16:15.661188 18172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:15.782492 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 17914
I20260811 12:16:15.811961 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 18073
I20260811 12:16:15.813048 18469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:15.937732 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 18073
2026-08-11T12:16:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12089 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:16:16 dist-test-slave-rmhg krb5kdc[19395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:16:16 dist-test-slave-rmhg krb5kdc[19395](info): set up 2 sockets
Aug 11 12:16:16 dist-test-slave-rmhg krb5kdc[19395](info): commencing operation
krb5kdc: starting...
W20260811 12:16:17.181936 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:16:17 dist-test-slave-rmhg krb5kdc[19395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:16:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.62.keytab.
Entry for principal oryx/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.62.keytab.
I20260811 12:16:17.456079 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:43987
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.62.keytab
--principal=oryx/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:43987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:17.812917 19412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:17.813323 19412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:17.813476 19412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:17.821054 19412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:17.821236 19412 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:16:17.821317 19412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:17.821408 19412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:17.821478 19412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:17.821573 19412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:17.832301 19412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:43987
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.62.keytab
--principal=oryx/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:43987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:17.833839 19412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:17.835320 19412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:17.847882 19425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:17.847858 19419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:17.847967 19420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:17.848601 19412 server_base.cc:1034] running on GCE node
I20260811 12:16:17.849488 19412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:17.850975 19412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:17.852248 19412 hybrid_clock.cc:648] HybridClock initialized: now 1786450577852174 us; error 43 us; skew 500 ppm
I20260811 12:16:17.852640 19412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:17 dist-test-slave-rmhg krb5kdc[19395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450577, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:17.855834 19412 init.cc:377] Logged in from keytab as oryx/127.7.39.62@KRBTEST.COM (short username oryx)
I20260811 12:16:17.857438 19412 webserver.cc:460] Webserver started at http://127.7.39.62:41689/ using document root <none> and password file <none>
I20260811 12:16:17.858274 19412 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:17.858402 19412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:17.858739 19412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:17.862047 19412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "475e0bae66f44457836d7734e1caa58d"
format_stamp: "Formatted at 2026-08-11 12:16:17 on dist-test-slave-rmhg"
server_key: "854f6064b6cae6f155f2255863cfb608"
server_key_iv: "20c2cd6a4aa2df75c09f48f0291b3eed"
server_key_version: "encryptionkey@0"
I20260811 12:16:17.862895 19412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "475e0bae66f44457836d7734e1caa58d"
format_stamp: "Formatted at 2026-08-11 12:16:17 on dist-test-slave-rmhg"
server_key: "854f6064b6cae6f155f2255863cfb608"
server_key_iv: "20c2cd6a4aa2df75c09f48f0291b3eed"
server_key_version: "encryptionkey@0"
I20260811 12:16:17.868477 19412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:16:17.872473 19437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:17.874090 19412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:17.874282 19412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "475e0bae66f44457836d7734e1caa58d"
format_stamp: "Formatted at 2026-08-11 12:16:17 on dist-test-slave-rmhg"
server_key: "854f6064b6cae6f155f2255863cfb608"
server_key_iv: "20c2cd6a4aa2df75c09f48f0291b3eed"
server_key_version: "encryptionkey@0"
I20260811 12:16:17.874501 19412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:17.890691 19412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:17.895346 19412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:17.895641 19412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:17.917281 19412 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:43987
I20260811 12:16:17.917300 19527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:43987 every 8 connection(s)
I20260811 12:16:17.918783 19412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:16:17.923004 19528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:17.924289 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 19412
I20260811 12:16:17.924541 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:16:17.925436 7324 external_mini_cluster.cc:1428] Setting key af654a4e9ce0ccdb7fd80f7249e59c22
I20260811 12:16:17.938556 19528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: Bootstrap starting.
Aug 11 12:16:17 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:17.942792 19528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:17.944308 19528 log.cc:824] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:17.948088 19528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: No bootstrap required, opened a new log
I20260811 12:16:17.951567 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:17.927685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.7.39.62:43987)
0811 12:16:17.928554 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:17.928567 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:17.928629 (+ 62us) server_negotiation.cc:407] Connection header received
0811 12:16:17.930387 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:17.930419 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:17.930915 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:17.931301 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:17.933258 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:17.934649 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:17.936518 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:17.937346 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:17.937490 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:17.942372 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:17.942388 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:17.942414 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:17.942490 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:17.946310 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:17.947203 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:17.947212 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:17.947221 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:17.947541 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:17.947952 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:17.947957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:17.947960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:17.948818 (+ 858us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:17.949128 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:17.949612 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 12:16:17.950097 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":413,"threads_started":1}
I20260811 12:16:17.958954 19528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475e0bae66f44457836d7734e1caa58d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43987 } }
I20260811 12:16:17.959373 19528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:17.959457 19528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475e0bae66f44457836d7734e1caa58d, State: Initialized, Role: FOLLOWER
I20260811 12:16:17.960037 19528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475e0bae66f44457836d7734e1caa58d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43987 } }
I20260811 12:16:17.960287 19528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:16:17.960395 19528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:16:17.960520 19528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:17.963055 19528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475e0bae66f44457836d7734e1caa58d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43987 } }
I20260811 12:16:17.963603 19528 leader_election.cc:302] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 475e0bae66f44457836d7734e1caa58d; no voters:
I20260811 12:16:17.964218 19528 leader_election.cc:288] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:16:17.964395 19541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:17.965641 19541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [term 1 LEADER]: Becoming Leader. State: Replica: 475e0bae66f44457836d7734e1caa58d, State: Running, Role: LEADER
I20260811 12:16:17.966256 19541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475e0bae66f44457836d7734e1caa58d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43987 } }
I20260811 12:16:17.966722 19528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:17.969005 19542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "475e0bae66f44457836d7734e1caa58d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475e0bae66f44457836d7734e1caa58d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43987 } } }
I20260811 12:16:17.969461 19542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:17.969365 19543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 475e0bae66f44457836d7734e1caa58d. Latest consensus state: current_term: 1 leader_uuid: "475e0bae66f44457836d7734e1caa58d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475e0bae66f44457836d7734e1caa58d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43987 } } }
I20260811 12:16:17.969627 19543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:17.970372 19552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:17.978035 19552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:17.988021 19552 catalog_manager.cc:1347] Generated new cluster ID: d09e1e2eb68545609030c3c595944703
I20260811 12:16:17.988178 19552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:18.007882 19552 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:16:18.009950 19552 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:18.019053 19552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: Generated new TSK 0
I20260811 12:16:18.019972 19552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab.
Entry for principal oryx/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab.
I20260811 12:16:18.084810 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43987
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab
--principal=oryx/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:18.446521 19573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:18.446863 19573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:18.447009 19573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:18.454818 19573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:18.454957 19573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:18.455084 19573 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:16:18.455200 19573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:18.465623 19573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab
--principal=oryx/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:43987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:18.467114 19573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:18.468673 19573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:18.482965 19581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:18.482983 19584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:18.483007 19580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:18.483866 19573 server_base.cc:1034] running on GCE node
I20260811 12:16:18.484579 19573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:18.485653 19573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:18.486907 19573 hybrid_clock.cc:648] HybridClock initialized: now 1786450578486826 us; error 52 us; skew 500 ppm
I20260811 12:16:18.487244 19573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:18 dist-test-slave-rmhg krb5kdc[19395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450578, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:18.490556 19573 init.cc:377] Logged in from keytab as oryx/127.7.39.1@KRBTEST.COM (short username oryx)
I20260811 12:16:18.492091 19573 webserver.cc:460] Webserver started at http://127.7.39.1:42157/ using document root <none> and password file <none>
I20260811 12:16:18.492832 19573 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:18.492944 19573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:18.493341 19573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:18.496474 19573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b"
format_stamp: "Formatted at 2026-08-11 12:16:18 on dist-test-slave-rmhg"
server_key: "7804ee2a897e5627cab6e4278bcc5efe"
server_key_iv: "831c1d611002269f091b3e6a54541c98"
server_key_version: "encryptionkey@0"
I20260811 12:16:18.497359 19573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b"
format_stamp: "Formatted at 2026-08-11 12:16:18 on dist-test-slave-rmhg"
server_key: "7804ee2a897e5627cab6e4278bcc5efe"
server_key_iv: "831c1d611002269f091b3e6a54541c98"
server_key_version: "encryptionkey@0"
I20260811 12:16:18.502904 19573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:16:18.506933 19592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:18.508512 19573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 12:16:18.508704 19573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b"
format_stamp: "Formatted at 2026-08-11 12:16:18 on dist-test-slave-rmhg"
server_key: "7804ee2a897e5627cab6e4278bcc5efe"
server_key_iv: "831c1d611002269f091b3e6a54541c98"
server_key_version: "encryptionkey@0"
I20260811 12:16:18.508958 19573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:18.523774 19573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:18.528280 19573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:18.528570 19573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:18.531569 19573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:18.531687 19573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:18.531831 19573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:18.531908 19573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:16:18 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450578, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:18.555917 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:18.532183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:44031 (local address 127.7.39.62:43987)
0811 12:16:18.532884 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:18.532888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:18.534066 (+ 1178us) server_negotiation.cc:407] Connection header received
0811 12:16:18.536175 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:18.536182 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:18.536269 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:18.536363 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:18.538790 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:18.539818 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:18.542338 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:18.542924 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:18.543074 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:18.548514 (+ 5440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:18.548524 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:18.548530 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:18.548584 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:18.552007 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:18.552853 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:18.552861 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:18.552864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:18.553044 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:18.553684 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:18.553688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:18.553691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:18.554004 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:18.554176 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:18.555451 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0811 12:16:18.555657 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":471,"threads_started":1}
I20260811 12:16:18.557891 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:18.532954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43987 (local address 127.7.39.1:44031)
0811 12:16:18.533897 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:18.533939 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:18.535861 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:18.536669 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:18.536681 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:18.537602 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:18.538571 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:18.538604 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:18.539978 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:18.539982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:18.542162 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:18.542181 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:18.543424 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:18.543436 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:18.543718 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:18.544701 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:18.544740 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:18.548160 (+ 3420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:18.552185 (+ 4025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:18.552197 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:18.552218 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:18.552704 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:18.553230 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:18.553236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:18.553239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:18.553495 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:18.554178 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:18.554193 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:18.554855 (+ 662us) client_negotiation.cc:769] Sending connection context
0811 12:16:18.555304 (+ 449us) client_negotiation.cc:240] Negotiation successful
0811 12:16:18.556007 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":384,"threads_started":1}
I20260811 12:16:18.580181 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:18.564008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:54817 (local address 127.7.39.62:43987)
0811 12:16:18.564222 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:18.564229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:18.564934 (+ 705us) server_negotiation.cc:407] Connection header received
0811 12:16:18.565124 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:18.565127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:18.565249 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:18.565345 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:18.566818 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:18.567846 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:18.569391 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:18.570022 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:18.570180 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:18.573264 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:18.573272 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:18.573277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:18.573328 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:18.576501 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:18.577292 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:18.577299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:18.577303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:18.577468 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:18.578286 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:18.578292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:18.578295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:18.578707 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:18.578879 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:18.579587 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 12:16:18.579822 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:16:18.580289 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:18.564017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43987 (local address 127.7.39.1:54817)
0811 12:16:18.564793 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:18.564818 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:18.565004 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:18.565424 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:18.565430 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:18.566152 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:18.566642 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:18.566675 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:18.568024 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:18.568033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:18.569118 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:18.569133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:18.570514 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:18.570519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:18.570680 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:18.571533 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:18.571555 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:18.572896 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:18.576672 (+ 3776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:18.576678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:18.576682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:18.577142 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:18.577781 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:18.577787 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:18.577857 (+ 70us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:18.578042 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:18.579031 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:18.579037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:18.579318 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 12:16:18.579471 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 12:16:18.579837 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":432,"threads_started":1}
I20260811 12:16:18.585459 19573 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:37457
I20260811 12:16:18.585513 19780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:37457 every 8 connection(s)
I20260811 12:16:18.586712 19573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:18.588932 19781 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43987
I20260811 12:16:18.589244 19781 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:18.589967 19781 heartbeater.cc:511] Master 127.7.39.62:43987 requested a full tablet report, sending...
I20260811 12:16:18.592088 19463 ts_manager.cc:194] Registered new tserver with Master: 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457)
I20260811 12:16:18.593693 19463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:44031
I20260811 12:16:18.595654 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 19573
I20260811 12:16:18.595878 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:16:18.596719 7324 external_mini_cluster.cc:1428] Setting key 522ec400a3547c0de09cce0da1e674d4
WARNING: no policy specified for oryx/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab.
Entry for principal oryx/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab.
I20260811 12:16:18.656876 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43987
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab
--principal=oryx/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:19.004279 19788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:19.004621 19788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:19.004765 19788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:19.012166 19788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:19.012343 19788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:19.012473 19788 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:16:19.012588 19788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:19.023027 19788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab
--principal=oryx/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:43987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:19.024384 19788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:19.025877 19788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:19.039614 19795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:19.039669 19798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:19.039710 19796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:19.040645 19788 server_base.cc:1034] running on GCE node
I20260811 12:16:19.041345 19788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:19.042323 19788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:19.043570 19788 hybrid_clock.cc:648] HybridClock initialized: now 1786450579043497 us; error 46 us; skew 500 ppm
I20260811 12:16:19.043885 19788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:19 dist-test-slave-rmhg krb5kdc[19395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:19.047348 19788 init.cc:377] Logged in from keytab as oryx/127.7.39.2@KRBTEST.COM (short username oryx)
I20260811 12:16:19.048905 19788 webserver.cc:460] Webserver started at http://127.7.39.2:45645/ using document root <none> and password file <none>
I20260811 12:16:19.049711 19788 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:19.049819 19788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:19.050163 19788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:19.053449 19788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "4794be8361c24aa6860e02cd3e520e4d"
format_stamp: "Formatted at 2026-08-11 12:16:19 on dist-test-slave-rmhg"
server_key: "586b652af8d5339badadf5a2411f5f26"
server_key_iv: "3618fe838b806b05dcf1b3c30f48c142"
server_key_version: "encryptionkey@0"
I20260811 12:16:19.054347 19788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "4794be8361c24aa6860e02cd3e520e4d"
format_stamp: "Formatted at 2026-08-11 12:16:19 on dist-test-slave-rmhg"
server_key: "586b652af8d5339badadf5a2411f5f26"
server_key_iv: "3618fe838b806b05dcf1b3c30f48c142"
server_key_version: "encryptionkey@0"
I20260811 12:16:19.060241 19788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 12:16:19.064248 19805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:19.065886 19788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:19.066077 19788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "4794be8361c24aa6860e02cd3e520e4d"
format_stamp: "Formatted at 2026-08-11 12:16:19 on dist-test-slave-rmhg"
server_key: "586b652af8d5339badadf5a2411f5f26"
server_key_iv: "3618fe838b806b05dcf1b3c30f48c142"
server_key_version: "encryptionkey@0"
I20260811 12:16:19.066311 19788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:19.078346 19788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:19.083416 19788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:19.083696 19788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:19.086683 19788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:19.086786 19788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:19.086964 19788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:19.087054 19788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:16:19 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:19.111915 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.087107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:60395 (local address 127.7.39.62:43987)
0811 12:16:19.087604 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.087621 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.089039 (+ 1418us) server_negotiation.cc:407] Connection header received
0811 12:16:19.091257 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.091264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.091373 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.091509 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:19.094075 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.095295 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.097872 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.098456 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.098596 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.103999 (+ 5403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:19.104006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:19.104009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:19.104051 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:19.107836 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.108774 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.108781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.108785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.108994 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.109581 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.109588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.109592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.110004 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:19.110162 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:19.111231 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0811 12:16:19.111502 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":236,"threads_started":1}
I20260811 12:16:19.113771 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.087823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43987 (local address 127.7.39.2:60395)
0811 12:16:19.088810 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:19.088878 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:19.090921 (+ 2043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:19.091828 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:19.091841 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:19.092739 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:19.093802 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.093834 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.095494 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.095500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.097674 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.097691 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.098929 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.098936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.099280 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.100169 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:19.100213 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:19.103676 (+ 3463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:19.108042 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:19.108053 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:19.108071 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:19.108619 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:19.109106 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:19.109118 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:19.109121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:19.109430 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:19.110169 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:19.110178 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:19.110680 (+ 502us) client_negotiation.cc:769] Sending connection context
0811 12:16:19.111074 (+ 394us) client_negotiation.cc:240] Negotiation successful
0811 12:16:19.111798 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":422,"threads_started":1}
I20260811 12:16:19.133927 19788 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:36753
I20260811 12:16:19.133940 19979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:36753 every 8 connection(s)
I20260811 12:16:19.135484 19788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:19.138088 19980 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43987
I20260811 12:16:19.138379 19980 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:19.139319 19980 heartbeater.cc:511] Master 127.7.39.62:43987 requested a full tablet report, sending...
I20260811 12:16:19.139389 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.124901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43987 (local address 127.7.39.2:34317)
0811 12:16:19.125755 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:19.125778 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:19.126032 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:19.126412 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:19.126417 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:19.126944 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:19.127261 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.127272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.128379 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.128385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.129559 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.129580 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.130806 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.130818 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.131054 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.131950 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:19.131969 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:19.132940 (+ 971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:19.136388 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:19.136395 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:19.136400 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:19.136873 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:19.137392 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:19.137398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:19.137402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:19.137605 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:19.138386 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:19.138390 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:19.138605 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 12:16:19.138738 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 12:16:19.139037 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":513,"threads_started":1}
I20260811 12:16:19.139484 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.124711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:34317 (local address 127.7.39.62:43987)
0811 12:16:19.124935 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.124942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.125903 (+ 961us) server_negotiation.cc:407] Connection header received
0811 12:16:19.126192 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.126197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.126269 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.126358 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:19.127443 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.128201 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.129814 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.130548 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.130717 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.133252 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:19.133259 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:19.133262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:19.133301 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:19.136208 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.137025 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.137032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.137035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.137244 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.137767 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.137772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.137774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.138253 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:19.138416 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:19.138879 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 12:16:19.139157 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:16:19.141032 19461 ts_manager.cc:194] Registered new tserver with Master: 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:19.142329 19461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:60395
I20260811 12:16:19.145052 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 19788
I20260811 12:16:19.145288 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:16:19.145773 7324 external_mini_cluster.cc:1428] Setting key 72414f00d2ff19b18787df886b35750c
WARNING: no policy specified for oryx/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab.
Entry for principal oryx/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab.
I20260811 12:16:19.206233 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43987
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab
--principal=oryx/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:19.557382 19984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:19.557698 19984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:19.557837 19984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:19.565408 19984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:19.565536 19984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:19.565665 19984 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:16:19.565773 19984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:19.576215 19984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab
--principal=oryx/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.39.62:43987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:19.577598 19984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:19.579066 19984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:19.594355 19991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:19.594380 19996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:19.594440 19994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:19.595124 19984 server_base.cc:1034] running on GCE node
I20260811 12:16:19.595774 19984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:19.596740 19984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:19.597265 19781 heartbeater.cc:503] Master 127.7.39.62:43987 was elected leader, sending a full tablet report...
I20260811 12:16:19.597957 19984 hybrid_clock.cc:648] HybridClock initialized: now 1786450579597881 us; error 52 us; skew 500 ppm
I20260811 12:16:19.598320 19984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:19 dist-test-slave-rmhg krb5kdc[19395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:19.601724 19984 init.cc:377] Logged in from keytab as oryx/127.7.39.3@KRBTEST.COM (short username oryx)
I20260811 12:16:19.603186 19984 webserver.cc:460] Webserver started at http://127.7.39.3:37485/ using document root <none> and password file <none>
I20260811 12:16:19.603890 19984 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:19.603996 19984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:19.604318 19984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:19.607357 19984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "0f2814848fd04b67adbf350a3971f6ee"
format_stamp: "Formatted at 2026-08-11 12:16:19 on dist-test-slave-rmhg"
server_key: "45b9707109ccc070592ebfcd67f8b2ee"
server_key_iv: "9da865144a11d6270526c584db50f5f7"
server_key_version: "encryptionkey@0"
I20260811 12:16:19.608145 19984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "0f2814848fd04b67adbf350a3971f6ee"
format_stamp: "Formatted at 2026-08-11 12:16:19 on dist-test-slave-rmhg"
server_key: "45b9707109ccc070592ebfcd67f8b2ee"
server_key_iv: "9da865144a11d6270526c584db50f5f7"
server_key_version: "encryptionkey@0"
I20260811 12:16:19.613457 19984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:16:19.616941 20011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:19.618340 19984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:19.618506 19984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "0f2814848fd04b67adbf350a3971f6ee"
format_stamp: "Formatted at 2026-08-11 12:16:19 on dist-test-slave-rmhg"
server_key: "45b9707109ccc070592ebfcd67f8b2ee"
server_key_iv: "9da865144a11d6270526c584db50f5f7"
server_key_version: "encryptionkey@0"
I20260811 12:16:19.618731 19984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:19.633000 19984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:19.637626 19984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:19.637895 19984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:19.640754 19984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:19.640846 19984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:19.640975 19984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:19.641036 19984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:16:19 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450579, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:19.667395 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.641464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:46861 (local address 127.7.39.62:43987)
0811 12:16:19.644236 (+ 2772us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.644243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.644268 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:16:19.646431 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.646437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.646507 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.646607 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:19.649354 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.650464 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.652895 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.653480 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.653626 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.657972 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:19.657981 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:19.657986 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:19.658033 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:19.661431 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.662632 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.662638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.662642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.662820 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.663501 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.663506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.663509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.663906 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:19.664050 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:19.666652 (+ 2602us) server_negotiation.cc:299] Negotiation successful
0811 12:16:19.666942 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2656,"thread_start_us":2515,"threads_started":1}
I20260811 12:16:19.667671 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.642245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43987 (local address 127.7.39.3:46861)
0811 12:16:19.643272 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:19.643337 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:19.646061 (+ 2724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:19.646911 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:19.646928 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:19.648041 (+ 1113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:19.649093 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.649124 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.650634 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.650643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.652724 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.652754 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.653955 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.653963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.654161 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.654906 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:19.654943 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:19.657713 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:19.661604 (+ 3891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:19.661614 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:19.661636 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:19.662424 (+ 788us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:19.662965 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:19.662971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:19.662975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:19.663327 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:19.664145 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:19.664155 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:19.664812 (+ 657us) client_negotiation.cc:769] Sending connection context
0811 12:16:19.665235 (+ 423us) client_negotiation.cc:240] Negotiation successful
0811 12:16:19.665987 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":431,"threads_started":1}
I20260811 12:16:19.688423 19984 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:32927
I20260811 12:16:19.688448 20213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:32927 every 8 connection(s)
I20260811 12:16:19.689491 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.674029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43987 (local address 127.7.39.3:58041)
0811 12:16:19.674727 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:19.674751 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:19.674920 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:19.675253 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:19.675258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:19.675911 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:19.676325 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.676339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.677475 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.677480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.679360 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.679381 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.680284 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.680293 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.680502 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.681326 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:19.681353 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:19.682493 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:19.686384 (+ 3891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:19.686390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:19.686394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:19.686951 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:19.687399 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:19.687404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:19.687408 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:19.687614 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:19.688371 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:19.688377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:19.688627 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 12:16:19.688792 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 12:16:19.689078 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":368,"threads_started":1}
I20260811 12:16:19.689564 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.673735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:58041 (local address 127.7.39.62:43987)
0811 12:16:19.673929 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.673936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.674856 (+ 920us) server_negotiation.cc:407] Connection header received
0811 12:16:19.675033 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.675037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.675122 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.675208 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:19.676483 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.677301 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.679567 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.680060 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.680170 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.682734 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:19.682743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:19.682748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:19.682804 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:19.686210 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.687074 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.687078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.687080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.687264 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.687783 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.687789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.687793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.688226 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:19.688385 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:19.688926 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 12:16:19.689229 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:16:19.690219 19984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:19.692809 20215 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43987
I20260811 12:16:19.693092 20215 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:19.693878 20215 heartbeater.cc:511] Master 127.7.39.62:43987 requested a full tablet report, sending...
I20260811 12:16:19.694326 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 19984
I20260811 12:16:19.694480 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:16:19.695032 7324 external_mini_cluster.cc:1428] Setting key 6f935a5b23e6ea5a730495e74dd298c4
I20260811 12:16:19.695370 19461 ts_manager.cc:194] Registered new tserver with Master: 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927)
I20260811 12:16:19.696383 19461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:46861
I20260811 12:16:19.698115 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:16:19 dist-test-slave-rmhg krb5kdc[19395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450579, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:16:19 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:19 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:16:19.761503 20028 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:19.750366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.7.39.62:43987)
0811 12:16:19.750627 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.750635 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.750833 (+ 198us) server_negotiation.cc:407] Connection header received
0811 12:16:19.751022 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.751026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.751115 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.751212 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:19.752616 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.753407 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.754715 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.755121 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.755220 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.761239 (+ 6019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53058: BlockingRecv error: recv got EOF from 127.0.0.1:53058 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 11 12:16:19 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450579, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:19.782564 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.764712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.7.39.62:43987)
0811 12:16:19.764915 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.764922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.765328 (+ 406us) server_negotiation.cc:407] Connection header received
0811 12:16:19.765535 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.765540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.765622 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.765711 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:19.766980 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.767797 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.769110 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.769659 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.769779 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.774056 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:19.774062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:19.774066 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:19.774108 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:19.779692 (+ 5584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.780430 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.780434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.780437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.780590 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:19.780925 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:19.780928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:19.780930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:19.781290 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:19.781545 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:19.781971 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 12:16:19.782249 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:16:19.787418 19461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:16:19.794627 19461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:16:19.827508 20241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.807968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:37457 (local address 127.0.0.1:60280)
0811 12:16:19.819616 (+ 11648us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:19.819638 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:19.819802 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:19.820682 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:19.820689 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:19.820719 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:19.821121 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.821137 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.822994 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.823004 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.825092 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.825117 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.826492 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.826498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.826678 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.826730 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 12:16:19.826841 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:16:19.827054 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11514,"mutex_wait_us":3,"thread_start_us":11717,"threads_started":1}
I20260811 12:16:19.828217 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.808862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.7.39.1:37457)
0811 12:16:19.817915 (+ 9053us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.817926 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.819945 (+ 2019us) server_negotiation.cc:407] Connection header received
0811 12:16:19.820083 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.820091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.820408 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.820644 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:19.821380 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.822824 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.825289 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.826275 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.826401 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.826658 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:19.827736 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0811 12:16:19.827928 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8840,"thread_start_us":8667,"threads_started":1}
I20260811 12:16:19.828578 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.808234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36753 (local address 127.0.0.1:60994)
0811 12:16:19.819937 (+ 11703us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:19.819966 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:19.820602 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:19.821558 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:19.821563 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:19.821590 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:19.821935 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.821944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.823505 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.823511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.825778 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.825793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.827780 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.827786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.828004 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.828038 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:16:19.828127 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:16:19.828259 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11581,"thread_start_us":9960,"threads_started":1}
I20260811 12:16:19.829187 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.809290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60994 (local address 127.7.39.2:36753)
0811 12:16:19.816605 (+ 7315us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.816615 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.820064 (+ 3449us) server_negotiation.cc:407] Connection header received
0811 12:16:19.820842 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.820852 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.821112 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.821318 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:19.822112 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.823366 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.825987 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.827316 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.827410 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.827604 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:19.828497 (+ 893us) server_negotiation.cc:299] Negotiation successful
0811 12:16:19.828860 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7127,"thread_start_us":6974,"threads_started":1}
I20260811 12:16:19.829698 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.807637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:32927 (local address 127.0.0.1:55964)
0811 12:16:19.818711 (+ 11074us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:19.818787 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:19.819045 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:19.821200 (+ 2155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:19.821205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:19.821229 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:19.821694 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.821707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.824147 (+ 2440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.824152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.826935 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:19.826949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.828981 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.828988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:19.829139 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.829194 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 12:16:19.829273 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:16:19.829403 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10711,"thread_start_us":11680,"threads_started":1}
I20260811 12:16:19.830044 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:19.807827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.7.39.3:32927)
0811 12:16:19.820618 (+ 12791us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:19.820626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:19.820659 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:16:19.820750 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:19.820757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:19.820983 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:19.821173 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:19.821926 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.823996 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.827132 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:19.828601 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:19.828726 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:19.828982 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:19.829557 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 12:16:19.829737 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12612,"thread_start_us":12501,"threads_started":1}
I20260811 12:16:19.832554 19878 tablet_service.cc:1467] Processing CreateTablet for tablet 88dc622a88324a7c942c5b58ac83b889 (DEFAULT_TABLE table=test-table [id=60092952bb3b40aebbf4d409df21c825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:19.832546 19670 tablet_service.cc:1467] Processing CreateTablet for tablet 88dc622a88324a7c942c5b58ac83b889 (DEFAULT_TABLE table=test-table [id=60092952bb3b40aebbf4d409df21c825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:19.834066 19878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88dc622a88324a7c942c5b58ac83b889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:19.834152 19670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88dc622a88324a7c942c5b58ac83b889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:19.833753 20102 tablet_service.cc:1467] Processing CreateTablet for tablet 88dc622a88324a7c942c5b58ac83b889 (DEFAULT_TABLE table=test-table [id=60092952bb3b40aebbf4d409df21c825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:19.835222 20102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88dc622a88324a7c942c5b58ac83b889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:19.847079 20250 tablet_bootstrap.cc:492] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Bootstrap starting.
I20260811 12:16:19.849519 20251 tablet_bootstrap.cc:492] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Bootstrap starting.
I20260811 12:16:19.851616 20250 tablet_bootstrap.cc:654] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:19.852396 20252 tablet_bootstrap.cc:492] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Bootstrap starting.
I20260811 12:16:19.853129 20251 tablet_bootstrap.cc:654] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:19.853485 20250 log.cc:824] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:19.854451 20251 log.cc:824] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:19.855655 20252 tablet_bootstrap.cc:654] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:19.856930 20250 tablet_bootstrap.cc:492] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: No bootstrap required, opened a new log
I20260811 12:16:19.856914 20252 log.cc:824] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:19.857272 20250 ts_tablet_manager.cc:1397] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 12:16:19.858775 20251 tablet_bootstrap.cc:492] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: No bootstrap required, opened a new log
I20260811 12:16:19.859088 20251 ts_tablet_manager.cc:1397] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 12:16:19.860378 20252 tablet_bootstrap.cc:492] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: No bootstrap required, opened a new log
I20260811 12:16:19.860654 20252 ts_tablet_manager.cc:1397] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:16:19.868216 20252 raft_consensus.cc:348] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:19.868132 20250 raft_consensus.cc:348] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:19.868634 20252 raft_consensus.cc:374] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:19.868666 20250 raft_consensus.cc:374] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:19.868727 20252 raft_consensus.cc:729] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2814848fd04b67adbf350a3971f6ee, State: Initialized, Role: FOLLOWER
I20260811 12:16:19.868754 20250 raft_consensus.cc:729] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4794be8361c24aa6860e02cd3e520e4d, State: Initialized, Role: FOLLOWER
I20260811 12:16:19.869411 20252 consensus_queue.cc:260] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:19.869486 20250 consensus_queue.cc:260] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:19.869652 20251 raft_consensus.cc:348] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:19.870226 20251 raft_consensus.cc:374] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:19.870330 20251 raft_consensus.cc:729] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb5b83b29e4c76bc4b1a783ac1b19b, State: Initialized, Role: FOLLOWER
I20260811 12:16:19.871176 20251 consensus_queue.cc:260] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:19.871562 20215 heartbeater.cc:503] Master 127.7.39.62:43987 was elected leader, sending a full tablet report...
I20260811 12:16:19.871798 20252 ts_tablet_manager.cc:1428] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 12:16:19.874697 20251 ts_tablet_manager.cc:1428] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 12:16:19.875782 19980 heartbeater.cc:503] Master 127.7.39.62:43987 was elected leader, sending a full tablet report...
I20260811 12:16:19.876446 20250 ts_tablet_manager.cc:1428] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
W20260811 12:16:19.888710 19981 tablet.cc:2369] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:16:19.942121 20216 tablet.cc:2369] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:16:20.040745 20264 raft_consensus.cc:482] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:20.041092 20264 raft_consensus.cc:504] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:20.042897 20264 leader_election.cc:288] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927), 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:20.048103 20263 raft_consensus.cc:482] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:20.048450 20263 raft_consensus.cc:504] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:20.050374 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.043490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:36945 (local address 127.7.39.2:36753)
0811 12:16:20.043730 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.043738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.044432 (+ 694us) server_negotiation.cc:407] Connection header received
0811 12:16:20.044627 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.044634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.044702 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.044773 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:20.045487 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.046253 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.048280 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.049343 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.049489 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.049601 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.049956 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.050113 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:16:20.050526 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.043508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:56697 (local address 127.7.39.3:32927)
0811 12:16:20.043768 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.043775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.044486 (+ 711us) server_negotiation.cc:407] Connection header received
0811 12:16:20.044650 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.044654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.044716 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.044799 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:20.045587 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.046312 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.048373 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.049545 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.049686 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.049790 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.050134 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.050286 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:16:20.050781 20263 leader_election.cc:288] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927), 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457)
I20260811 12:16:20.050854 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.043199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36753 (local address 127.7.39.1:36945)
0811 12:16:20.044297 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.044313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.044476 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.044814 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.044819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.044842 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:20.045340 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.045351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.046384 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.046389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.048132 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.048146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.049559 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.049565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.049756 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.049814 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.049912 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.050074 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":4,"thread_start_us":827,"threads_started":1}
I20260811 12:16:20.051460 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.043316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:32927 (local address 127.7.39.1:56697)
0811 12:16:20.044346 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.044361 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.044520 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.044835 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.044840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.044863 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:20.045418 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.045433 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.046454 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.046460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.048169 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.048185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.049768 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.049774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.049956 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.049986 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.050709 (+ 723us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.050848 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":49,"thread_start_us":611,"threads_started":1}
I20260811 12:16:20.052516 20133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88dc622a88324a7c942c5b58ac83b889" candidate_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f2814848fd04b67adbf350a3971f6ee" is_pre_election: true
I20260811 12:16:20.053215 20133 raft_consensus.cc:2454] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in term 0.
I20260811 12:16:20.053252 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88dc622a88324a7c942c5b58ac83b889" candidate_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d" is_pre_election: true
I20260811 12:16:20.053779 19910 raft_consensus.cc:2454] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in term 0.
I20260811 12:16:20.054036 19596 leader_election.cc:302] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 51bb5b83b29e4c76bc4b1a783ac1b19b; no voters:
I20260811 12:16:20.054474 20264 raft_consensus.cc:2781] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:20.054620 20264 raft_consensus.cc:482] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:20.054724 20264 raft_consensus.cc:3037] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.057051 20264 raft_consensus.cc:504] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:20.058295 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88dc622a88324a7c942c5b58ac83b889" candidate_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d"
I20260811 12:16:20.058516 19910 raft_consensus.cc:3037] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.058769 20133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88dc622a88324a7c942c5b58ac83b889" candidate_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f2814848fd04b67adbf350a3971f6ee"
I20260811 12:16:20.058997 20133 raft_consensus.cc:3037] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.060405 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.051208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:54883 (local address 127.7.39.1:37457)
0811 12:16:20.051827 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.051832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.052281 (+ 449us) server_negotiation.cc:407] Connection header received
0811 12:16:20.052576 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.052582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.052662 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.052731 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:20.053777 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.054896 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.057841 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.059034 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.059147 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.059245 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.060009 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.060167 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474}
I20260811 12:16:20.060437 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.051902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:59321 (local address 127.7.39.3:32927)
0811 12:16:20.052063 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.052067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.052377 (+ 310us) server_negotiation.cc:407] Connection header received
0811 12:16:20.052590 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.052594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.052666 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.052731 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:20.053873 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.054935 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.057825 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.059057 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.059146 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.059239 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.060024 (+ 785us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.060160 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:16:20.060742 20264 leader_election.cc:288] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [CANDIDATE]: Term 1 election: Requested vote from peers 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927), 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:20.061339 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.051060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:37457 (local address 127.7.39.2:54883)
0811 12:16:20.052182 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.052200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.052400 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.052938 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.052943 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.052973 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:20.053589 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.053602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.055091 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.055096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.057668 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.057677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.059527 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.059535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.059824 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.059852 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.060621 (+ 769us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.060780 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":193,"thread_start_us":1270,"threads_started":1}
I20260811 12:16:20.061363 20268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.051461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:32927 (local address 127.7.39.2:59321)
0811 12:16:20.052297 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.052311 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.052453 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.052993 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.052997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.053016 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:20.053702 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.053715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.055140 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.055148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.057624 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.057642 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.059533 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.059538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.059774 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.059837 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.060621 (+ 784us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.060780 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":155,"thread_start_us":601,"threads_started":1}
I20260811 12:16:20.062279 20132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88dc622a88324a7c942c5b58ac83b889" candidate_uuid: "4794be8361c24aa6860e02cd3e520e4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f2814848fd04b67adbf350a3971f6ee" is_pre_election: true
I20260811 12:16:20.062460 19713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88dc622a88324a7c942c5b58ac83b889" candidate_uuid: "4794be8361c24aa6860e02cd3e520e4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" is_pre_election: true
I20260811 12:16:20.062933 19713 raft_consensus.cc:2379] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4794be8361c24aa6860e02cd3e520e4d in current term 1: Already voted for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in this term.
I20260811 12:16:20.063747 19806 leader_election.cc:302] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4794be8361c24aa6860e02cd3e520e4d; no voters: 0f2814848fd04b67adbf350a3971f6ee, 51bb5b83b29e4c76bc4b1a783ac1b19b
I20260811 12:16:20.063941 20133 raft_consensus.cc:2454] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in term 1.
I20260811 12:16:20.064280 19910 raft_consensus.cc:2454] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in term 1.
I20260811 12:16:20.064402 19596 leader_election.cc:302] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 51bb5b83b29e4c76bc4b1a783ac1b19b; no voters:
I20260811 12:16:20.064572 20263 raft_consensus.cc:2726] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 12:16:20.064818 20264 raft_consensus.cc:2781] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:20.065286 20264 raft_consensus.cc:686] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 LEADER]: Becoming Leader. State: Replica: 51bb5b83b29e4c76bc4b1a783ac1b19b, State: Running, Role: LEADER
I20260811 12:16:20.065855 20264 consensus_queue.cc:237] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } }
I20260811 12:16:20.070817 19460 catalog_manager.cc:5495] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b reported cstate change: term changed from 0 to 1, leader changed from <none> to 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1). New cstate: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } health_report { overall_health: UNKNOWN } } }
W20260811 12:16:20.089445 19782 tablet.cc:2369] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:16:20.133483 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.127974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.7.39.1:37457)
0811 12:16:20.128198 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.128205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.128242 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:16:20.128522 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.128526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.128618 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.128761 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:20.129134 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.129913 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.131265 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.131684 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.131797 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.132242 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:20.132833 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:20.132970 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.133086 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.133198 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:16:20.141671 19910 raft_consensus.cc:1261] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Refusing update from remote peer 51bb5b83b29e4c76bc4b1a783ac1b19b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:20.141696 20133 raft_consensus.cc:1261] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Refusing update from remote peer 51bb5b83b29e4c76bc4b1a783ac1b19b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:20.142596 20264 consensus_queue.cc:1035] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:20.142946 20269 consensus_queue.cc:1035] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:20.155365 20274 mvcc.cc:205] Tried to move back new op lower bound from 7317301576249974784 to 7317301575953997824. Current Snapshot: MvccSnapshot[applied={T|T < 7317301576249974784}]
I20260811 12:16:20.172102 19460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:20.172407 19460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:20.175987 19460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: Sending DeleteTablet for 3 replicas of tablet 88dc622a88324a7c942c5b58ac83b889
I20260811 12:16:20.177210 20102 tablet_service.cc:1514] Processing DeleteTablet for tablet 88dc622a88324a7c942c5b58ac83b889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:20 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:55964
I20260811 12:16:20.177241 19878 tablet_service.cc:1514] Processing DeleteTablet for tablet 88dc622a88324a7c942c5b58ac83b889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:20 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:60994
I20260811 12:16:20.177999 19670 tablet_service.cc:1514] Processing DeleteTablet for tablet 88dc622a88324a7c942c5b58ac83b889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:20 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:60280
I20260811 12:16:20.178231 20292 tablet_replica.cc:330] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: stopping tablet replica
I20260811 12:16:20.178817 20292 raft_consensus.cc:2229] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:20.178908 20294 tablet_replica.cc:330] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: stopping tablet replica
I20260811 12:16:20.179351 20292 raft_consensus.cc:2258] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:20.179447 20294 raft_consensus.cc:2229] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:16:20.180483 20294 raft_consensus.cc:2258] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:20.180371 19460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:16:20.181073 19460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:16:20.181463 20292 ts_tablet_manager.cc:1905] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:20.182659 20294 ts_tablet_manager.cc:1905] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:20.188684 20292 ts_tablet_manager.cc:1918] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:20.188877 20292 log.cc:1196] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/88dc622a88324a7c942c5b58ac83b889
I20260811 12:16:20.189491 20292 ts_tablet_manager.cc:1939] T 88dc622a88324a7c942c5b58ac83b889 P 4794be8361c24aa6860e02cd3e520e4d: Deleting consensus metadata
I20260811 12:16:20.190201 20294 ts_tablet_manager.cc:1918] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:20.190402 20294 log.cc:1196] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/88dc622a88324a7c942c5b58ac83b889
I20260811 12:16:20.190929 20294 ts_tablet_manager.cc:1939] T 88dc622a88324a7c942c5b58ac83b889 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting consensus metadata
I20260811 12:16:20.191603 19442 catalog_manager.cc:4841] TS 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753): tablet 88dc622a88324a7c942c5b58ac83b889 (table test-table [id=60092952bb3b40aebbf4d409df21c825]) successfully deleted
I20260811 12:16:20.193248 20101 tablet_service.cc:1467] Processing CreateTablet for tablet c084d360e0034a40bb795c3c9161c4bc (DEFAULT_TABLE table=test-table [id=e0b1c87843c44128a90f1c7592b61ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:20.193271 19878 tablet_service.cc:1467] Processing CreateTablet for tablet c084d360e0034a40bb795c3c9161c4bc (DEFAULT_TABLE table=test-table [id=e0b1c87843c44128a90f1c7592b61ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:20.193724 20101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c084d360e0034a40bb795c3c9161c4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:20.193748 19878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c084d360e0034a40bb795c3c9161c4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:20.194092 19438 catalog_manager.cc:4841] TS 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457): tablet 88dc622a88324a7c942c5b58ac83b889 (table test-table [id=60092952bb3b40aebbf4d409df21c825]) successfully deleted
I20260811 12:16:20.197633 19670 tablet_service.cc:1467] Processing CreateTablet for tablet c084d360e0034a40bb795c3c9161c4bc (DEFAULT_TABLE table=test-table [id=e0b1c87843c44128a90f1c7592b61ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:20.198151 19670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c084d360e0034a40bb795c3c9161c4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:20.199986 20252 tablet_bootstrap.cc:492] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: Bootstrap starting.
I20260811 12:16:20.201721 20252 tablet_bootstrap.cc:654] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:20.202876 20250 tablet_bootstrap.cc:492] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: Bootstrap starting.
I20260811 12:16:20.204542 20250 tablet_bootstrap.cc:654] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:20.205437 20251 tablet_bootstrap.cc:492] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: Bootstrap starting.
I20260811 12:16:20.207211 20251 tablet_bootstrap.cc:654] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:20.210124 20293 tablet_replica.cc:330] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: stopping tablet replica
I20260811 12:16:20.210650 20293 raft_consensus.cc:2229] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:20.211156 20293 raft_consensus.cc:2258] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:20.213621 20293 ts_tablet_manager.cc:1905] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:20.220614 20251 tablet_bootstrap.cc:492] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: No bootstrap required, opened a new log
I20260811 12:16:20.220813 20251 ts_tablet_manager.cc:1397] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20260811 12:16:20.221076 20293 ts_tablet_manager.cc:1918] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:20.221546 20293 log.cc:1196] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/88dc622a88324a7c942c5b58ac83b889
I20260811 12:16:20.221802 20251 raft_consensus.cc:348] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.222143 20251 raft_consensus.cc:374] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:20.222143 20293 ts_tablet_manager.cc:1939] T 88dc622a88324a7c942c5b58ac83b889 P 0f2814848fd04b67adbf350a3971f6ee: Deleting consensus metadata
I20260811 12:16:20.222245 20251 raft_consensus.cc:729] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb5b83b29e4c76bc4b1a783ac1b19b, State: Initialized, Role: FOLLOWER
I20260811 12:16:20.222577 20251 consensus_queue.cc:260] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.223290 20251 ts_tablet_manager.cc:1428] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 12:16:20.223997 19443 catalog_manager.cc:4841] TS 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927): tablet 88dc622a88324a7c942c5b58ac83b889 (table test-table [id=60092952bb3b40aebbf4d409df21c825]) successfully deleted
I20260811 12:16:20.225785 20250 tablet_bootstrap.cc:492] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: No bootstrap required, opened a new log
I20260811 12:16:20.225951 20252 tablet_bootstrap.cc:492] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: No bootstrap required, opened a new log
I20260811 12:16:20.226069 20250 ts_tablet_manager.cc:1397] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.007s
I20260811 12:16:20.226095 20252 ts_tablet_manager.cc:1397] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.007s
I20260811 12:16:20.226909 20252 raft_consensus.cc:348] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.226910 20250 raft_consensus.cc:348] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.227154 20252 raft_consensus.cc:374] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:20.227253 20252 raft_consensus.cc:729] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2814848fd04b67adbf350a3971f6ee, State: Initialized, Role: FOLLOWER
I20260811 12:16:20.227276 20250 raft_consensus.cc:374] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:20.227452 20250 raft_consensus.cc:729] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4794be8361c24aa6860e02cd3e520e4d, State: Initialized, Role: FOLLOWER
I20260811 12:16:20.227541 20252 consensus_queue.cc:260] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.227794 20250 consensus_queue.cc:260] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.228246 20252 ts_tablet_manager.cc:1428] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 12:16:20.229502 20250 ts_tablet_manager.cc:1428] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:16:20.353211 20259 raft_consensus.cc:482] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:20.353466 20259 raft_consensus.cc:504] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.355023 20259 leader_election.cc:288] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457), 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:20.362998 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.355675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:36925 (local address 127.7.39.2:36753)
0811 12:16:20.355883 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.355891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.356490 (+ 599us) server_negotiation.cc:407] Connection header received
0811 12:16:20.356779 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.356784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.356867 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.356948 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:20.357497 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.358362 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.360738 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.362002 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.362152 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.362255 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.362564 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.362698 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:16:20.363305 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.355709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:52677 (local address 127.7.39.1:37457)
0811 12:16:20.355900 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.355906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.356807 (+ 901us) server_negotiation.cc:407] Connection header received
0811 12:16:20.356978 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.356983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.357080 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.357206 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:20.357861 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.358875 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.361140 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.362152 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.362286 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.362383 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.362872 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.363045 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:16:20.363356 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.355345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36753 (local address 127.7.39.3:36925)
0811 12:16:20.356361 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.356384 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.356619 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.356988 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.356992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.357014 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:20.357322 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.357337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.358502 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.358508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.360553 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.360571 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.362187 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.362194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.362375 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.362414 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.362508 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.362665 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":21,"thread_start_us":1037,"threads_started":1}
I20260811 12:16:20.364107 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.355363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:37457 (local address 127.7.39.3:52677)
0811 12:16:20.356695 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.356713 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.356842 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.357238 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.357243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.357268 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:20.357678 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.357690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.359035 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.359041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.360973 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.360985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.362459 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.362466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.362670 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.362706 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.363620 (+ 914us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.363765 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"mutex_wait_us":1,"thread_start_us":665,"threads_started":1}
I20260811 12:16:20.364145 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c084d360e0034a40bb795c3c9161c4bc" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d" is_pre_election: true
I20260811 12:16:20.364450 19910 raft_consensus.cc:2454] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 0.
I20260811 12:16:20.364760 19713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c084d360e0034a40bb795c3c9161c4bc" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" is_pre_election: true
I20260811 12:16:20.364903 20019 leader_election.cc:302] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 4794be8361c24aa6860e02cd3e520e4d; no voters:
I20260811 12:16:20.365036 19713 raft_consensus.cc:2454] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 0.
I20260811 12:16:20.365365 20259 raft_consensus.cc:2781] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:20.365502 20259 raft_consensus.cc:482] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:20.365597 20259 raft_consensus.cc:3037] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.368009 20259 raft_consensus.cc:504] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.368748 20259 leader_election.cc:288] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 election: Requested vote from peers 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457), 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:20.369223 19713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c084d360e0034a40bb795c3c9161c4bc" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b"
I20260811 12:16:20.369462 19713 raft_consensus.cc:3037] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.369866 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c084d360e0034a40bb795c3c9161c4bc" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d"
I20260811 12:16:20.370067 19910 raft_consensus.cc:3037] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.372475 19910 raft_consensus.cc:2454] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 1.
I20260811 12:16:20.372859 20019 leader_election.cc:302] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 4794be8361c24aa6860e02cd3e520e4d; no voters:
I20260811 12:16:20.372859 19713 raft_consensus.cc:2454] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 1.
I20260811 12:16:20.373215 20259 raft_consensus.cc:2781] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:20.373796 20259 raft_consensus.cc:686] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 1 LEADER]: Becoming Leader. State: Replica: 0f2814848fd04b67adbf350a3971f6ee, State: Running, Role: LEADER
I20260811 12:16:20.374222 20259 consensus_queue.cc:237] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.380030 19463 catalog_manager.cc:5495] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3). New cstate: current_term: 1 leader_uuid: "0f2814848fd04b67adbf350a3971f6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } health_report { overall_health: HEALTHY } } }
Aug 11 12:16:20 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450577, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.62@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:20.448405 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.431743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53084 (local address 127.7.39.62:43987)
0811 12:16:20.432719 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.432724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.432746 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:16:20.432793 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.432796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.432867 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.432962 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:20.434535 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.435313 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.437186 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.437627 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.437727 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.442120 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:20.442125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:20.442129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:20.442164 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:20.444978 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:20.446077 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:20.446082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:20.446084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:20.446227 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:20.446684 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:20.446688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:20.446690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:20.446965 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:20.447095 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.447754 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.448099 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":874,"thread_start_us":732,"threads_started":1}
I20260811 12:16:20.448454 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.431645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43987 (local address 127.0.0.1:53084)
0811 12:16:20.432383 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.432407 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.432604 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.433061 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.433066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.434063 (+ 997us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:20.434352 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.434368 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.435470 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.435475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.436987 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.437006 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.437823 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.437830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.438034 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.438803 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:20.438827 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:20.441891 (+ 3064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:20.445146 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:20.445196 (+ 50us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:20.445206 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:20.445939 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:20.446350 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:20.446356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:20.446360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:20.446557 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:20.447111 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:20.447122 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:20.447340 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.447616 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.447982 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":356,"threads_started":1}
I20260811 12:16:20.454298 19463 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:53084:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:16:20.455430 19463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:16:20.463003 19670 tablet_service.cc:1467] Processing CreateTablet for tablet 750a58037d2640d6a01630d114115b01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bbc8e07607014807be7f337782176523]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:16:20.463016 19878 tablet_service.cc:1467] Processing CreateTablet for tablet 750a58037d2640d6a01630d114115b01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bbc8e07607014807be7f337782176523]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:16:20.463145 20101 tablet_service.cc:1467] Processing CreateTablet for tablet 750a58037d2640d6a01630d114115b01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bbc8e07607014807be7f337782176523]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:16:20.463621 19670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750a58037d2640d6a01630d114115b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:20.463636 19878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750a58037d2640d6a01630d114115b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:20.463636 20101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750a58037d2640d6a01630d114115b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:20.469741 20251 tablet_bootstrap.cc:492] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Bootstrap starting.
I20260811 12:16:20.470847 20250 tablet_bootstrap.cc:492] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: Bootstrap starting.
I20260811 12:16:20.471563 20251 tablet_bootstrap.cc:654] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:20.472384 20250 tablet_bootstrap.cc:654] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:20.472810 20252 tablet_bootstrap.cc:492] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: Bootstrap starting.
I20260811 12:16:20.474622 20252 tablet_bootstrap.cc:654] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:20.475613 20250 tablet_bootstrap.cc:492] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: No bootstrap required, opened a new log
I20260811 12:16:20.475793 20250 ts_tablet_manager.cc:1397] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260811 12:16:20.476812 20250 raft_consensus.cc:348] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.477046 20251 tablet_bootstrap.cc:492] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: No bootstrap required, opened a new log
I20260811 12:16:20.477085 20250 raft_consensus.cc:374] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:20.477195 20250 raft_consensus.cc:729] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4794be8361c24aa6860e02cd3e520e4d, State: Initialized, Role: FOLLOWER
I20260811 12:16:20.477284 20251 ts_tablet_manager.cc:1397] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 12:16:20.477463 20250 consensus_queue.cc:260] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.478091 20250 ts_tablet_manager.cc:1428] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:16:20.478456 20251 raft_consensus.cc:348] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.478787 20251 raft_consensus.cc:374] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:20.478879 20251 raft_consensus.cc:729] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb5b83b29e4c76bc4b1a783ac1b19b, State: Initialized, Role: FOLLOWER
I20260811 12:16:20.479228 20251 consensus_queue.cc:260] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.479733 20264 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.480039 20264 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:16:20.480811 20263 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.481097 20263 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:16:20.483361 20251 ts_tablet_manager.cc:1428] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 12:16:20.486788 20252 tablet_bootstrap.cc:492] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: No bootstrap required, opened a new log
I20260811 12:16:20.486994 20252 ts_tablet_manager.cc:1397] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260811 12:16:20.487854 20252 raft_consensus.cc:348] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.488129 20252 raft_consensus.cc:374] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:20.488224 20252 raft_consensus.cc:729] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2814848fd04b67adbf350a3971f6ee, State: Initialized, Role: FOLLOWER
I20260811 12:16:20.488490 20252 consensus_queue.cc:260] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.489024 20259 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.489365 20259 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:16:20.489877 20252 ts_tablet_manager.cc:1428] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 12:16:20.638561 20264 raft_consensus.cc:482] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:20.638770 20264 raft_consensus.cc:504] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.639453 20264 leader_election.cc:288] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753), 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927)
I20260811 12:16:20.640035 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750a58037d2640d6a01630d114115b01" candidate_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d" is_pre_election: true
I20260811 12:16:20.640055 20133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750a58037d2640d6a01630d114115b01" candidate_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f2814848fd04b67adbf350a3971f6ee" is_pre_election: true
I20260811 12:16:20.640283 19910 raft_consensus.cc:2454] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in term 0.
I20260811 12:16:20.640307 20133 raft_consensus.cc:2454] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in term 0.
I20260811 12:16:20.640769 19596 leader_election.cc:302] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 51bb5b83b29e4c76bc4b1a783ac1b19b; no voters:
I20260811 12:16:20.641055 20264 raft_consensus.cc:2781] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:20.641207 20264 raft_consensus.cc:482] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:20.641299 20264 raft_consensus.cc:3037] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.643404 20264 raft_consensus.cc:504] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.644049 20264 leader_election.cc:288] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [CANDIDATE]: Term 1 election: Requested vote from peers 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753), 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927)
I20260811 12:16:20.644562 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750a58037d2640d6a01630d114115b01" candidate_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d"
I20260811 12:16:20.644614 20133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750a58037d2640d6a01630d114115b01" candidate_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f2814848fd04b67adbf350a3971f6ee"
I20260811 12:16:20.644785 19910 raft_consensus.cc:3037] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.644809 20133 raft_consensus.cc:3037] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:20.646819 20133 raft_consensus.cc:2454] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in term 1.
I20260811 12:16:20.646869 19910 raft_consensus.cc:2454] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51bb5b83b29e4c76bc4b1a783ac1b19b in term 1.
I20260811 12:16:20.647302 19596 leader_election.cc:302] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 4794be8361c24aa6860e02cd3e520e4d, 51bb5b83b29e4c76bc4b1a783ac1b19b; no voters:
I20260811 12:16:20.647606 20264 raft_consensus.cc:2781] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:20.647781 20264 raft_consensus.cc:686] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 LEADER]: Becoming Leader. State: Replica: 51bb5b83b29e4c76bc4b1a783ac1b19b, State: Running, Role: LEADER
I20260811 12:16:20.648073 20264 consensus_queue.cc:237] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:20.649700 20269 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: TxnStatusTablet state changed. Reason: New leader 51bb5b83b29e4c76bc4b1a783ac1b19b. Latest consensus state: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.650040 20269 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:16:20.650672 20325 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:16:20.650918 20325 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:16:20.651312 19463 catalog_manager.cc:5495] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b reported cstate change: term changed from 0 to 1, leader changed from <none> to 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1). New cstate: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } health_report { overall_health: UNKNOWN } } }
I20260811 12:16:20.678158 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.672900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:37457 (local address 127.0.0.1:60302)
0811 12:16:20.673208 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.673233 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.673360 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.673907 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.673912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.673924 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:16:20.674094 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.674103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.675103 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.675109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.676100 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.676112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.676892 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.676899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.677068 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.677130 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:16:20.677588 (+ 458us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:16:20.677610 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.677689 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.677853 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 12:16:20.678205 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.672923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60302 (local address 127.7.39.1:37457)
0811 12:16:20.673600 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.673627 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.673644 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:16:20.673706 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.673709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.673775 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.673855 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:20.674234 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.674970 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.676243 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.676732 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.676856 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.677354 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:20.677482 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:20.677560 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.677753 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.677857 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563}
I20260811 12:16:20.683053 20133 raft_consensus.cc:1261] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Refusing update from remote peer 51bb5b83b29e4c76bc4b1a783ac1b19b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:20.683161 19910 raft_consensus.cc:1261] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Refusing update from remote peer 51bb5b83b29e4c76bc4b1a783ac1b19b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:20.683784 20269 consensus_queue.cc:1035] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:20.684144 20264 consensus_queue.cc:1035] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:20.701977 20263 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: TxnStatusTablet state changed. Reason: New leader 51bb5b83b29e4c76bc4b1a783ac1b19b. Latest consensus state: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.702275 20263 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:16:20.707378 20259 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: TxnStatusTablet state changed. Reason: New leader 51bb5b83b29e4c76bc4b1a783ac1b19b. Latest consensus state: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.707746 20259 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:16:20.708900 20264 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.709270 20264 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:16:20.710099 20328 mvcc.cc:205] Tried to move back new op lower bound from 7317301578471448576 to 7317301578336845824. Current Snapshot: MvccSnapshot[applied={T|T < 7317301578471448576}]
I20260811 12:16:20.712203 20269 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.712440 20269 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 51bb5b83b29e4c76bc4b1a783ac1b19b: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:16:20.724133 20263 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.724444 20263 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 4794be8361c24aa6860e02cd3e520e4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:16:20.729875 20259 tablet_replica.cc:439] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } } }
I20260811 12:16:20.730217 20259 tablet_replica.cc:442] T 750a58037d2640d6a01630d114115b01 P 0f2814848fd04b67adbf350a3971f6ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:16:20.742936 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.735281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.7.39.3:32927)
0811 12:16:20.737561 (+ 2280us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.737566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.737583 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:16:20.737665 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.737668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.737756 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.737918 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:20.738409 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.739221 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.740651 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.741076 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.741214 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.741675 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:20.742257 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:20.742394 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.742513 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.742600 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2177,"thread_start_us":2061,"threads_started":1}
I20260811 12:16:20.759784 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.754088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:40167 (local address 127.7.39.1:37457)
0811 12:16:20.754326 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.754334 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.754950 (+ 616us) server_negotiation.cc:407] Connection header received
0811 12:16:20.755152 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.755157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.755227 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.755312 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:20.755775 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.756551 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.758017 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.758447 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.758580 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.759004 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:20.759133 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:20.759210 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.759366 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.759446 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:16:20.759840 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.754015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:37457 (local address 127.7.39.3:40167)
0811 12:16:20.754841 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.754858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.755017 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.755329 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.755334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.755345 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:16:20.755621 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.755631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.756702 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.756708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.757866 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.757880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.758628 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.758635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.758760 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.758821 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:16:20.759228 (+ 407us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:16:20.759248 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.759317 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.759458 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":498,"threads_started":1}
I20260811 12:16:20.771476 19910 raft_consensus.cc:1261] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Refusing update from remote peer 0f2814848fd04b67adbf350a3971f6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:20.771732 19713 raft_consensus.cc:1261] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Refusing update from remote peer 0f2814848fd04b67adbf350a3971f6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:20.772277 20305 consensus_queue.cc:1035] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:20.772758 20259 consensus_queue.cc:1035] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:20.779181 20329 mvcc.cc:205] Tried to move back new op lower bound from 7317301578832171008 to 7317301577216753664. Current Snapshot: MvccSnapshot[applied={T|T < 7317301578832171008}]
I20260811 12:16:20.816860 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.806955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:47639 (local address 127.7.39.3:32927)
0811 12:16:20.807144 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.807151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.809947 (+ 2796us) server_negotiation.cc:407] Connection header received
0811 12:16:20.810171 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.810177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.810269 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.810350 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:20.811081 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.812201 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.814603 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.815032 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.815144 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.815917 (+ 773us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:20.816064 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:20.816175 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:20.816398 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 12:16:20.816530 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:16:20.816991 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:20.806798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:32927 (local address 127.7.39.1:47639)
0811 12:16:20.809794 (+ 2996us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:20.809818 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:20.809984 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:20.810393 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:20.810398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:20.810412 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:16:20.810855 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.810868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.812378 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.812384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.814387 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:20.814404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:20.815225 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:20.815232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:20.815434 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:20.815548 (+ 114us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:16:20.816239 (+ 691us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:16:20.816253 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 12:16:20.816346 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:16:20.816507 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2747,"thread_start_us":2553,"threads_started":1}
I20260811 12:16:20.861899 19460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:20.862195 19460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:20.866294 19460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: Sending DeleteTablet for 3 replicas of tablet c084d360e0034a40bb795c3c9161c4bc
I20260811 12:16:20.867583 19878 tablet_service.cc:1514] Processing DeleteTablet for tablet c084d360e0034a40bb795c3c9161c4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:20 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:60994
I20260811 12:16:20.868252 20362 tablet_replica.cc:330] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: stopping tablet replica
I20260811 12:16:20.868458 20362 raft_consensus.cc:2229] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:20.868698 20362 raft_consensus.cc:2258] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:20.869948 20362 ts_tablet_manager.cc:1905] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:20.873013 20101 tablet_service.cc:1514] Processing DeleteTablet for tablet c084d360e0034a40bb795c3c9161c4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:20 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:55964
I20260811 12:16:20.873836 19670 tablet_service.cc:1514] Processing DeleteTablet for tablet c084d360e0034a40bb795c3c9161c4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:20 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:60280
I20260811 12:16:20.877897 20362 ts_tablet_manager.cc:1918] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:16:20.878118 20362 log.cc:1196] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/c084d360e0034a40bb795c3c9161c4bc
I20260811 12:16:20.878712 20362 ts_tablet_manager.cc:1939] T c084d360e0034a40bb795c3c9161c4bc P 4794be8361c24aa6860e02cd3e520e4d: Deleting consensus metadata
I20260811 12:16:20.880043 19442 catalog_manager.cc:4841] TS 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753): tablet c084d360e0034a40bb795c3c9161c4bc (table test-table [id=e0b1c87843c44128a90f1c7592b61ad2]) successfully deleted
I20260811 12:16:20.887842 20363 tablet_replica.cc:330] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: stopping tablet replica
I20260811 12:16:20.888550 20364 tablet_replica.cc:330] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: stopping tablet replica
I20260811 12:16:20.888880 20363 raft_consensus.cc:2229] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:16:20.889605 20363 raft_consensus.cc:2258] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:20.889914 20364 raft_consensus.cc:2229] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:20.890393 20364 raft_consensus.cc:2258] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:20.892438 20364 ts_tablet_manager.cc:1905] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:20.893335 20363 ts_tablet_manager.cc:1905] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:20.902352 20363 ts_tablet_manager.cc:1918] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:16:20.909142 20364 ts_tablet_manager.cc:1918] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:16:20.909502 20363 log.cc:1196] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/c084d360e0034a40bb795c3c9161c4bc
I20260811 12:16:20.910061 20363 ts_tablet_manager.cc:1939] T c084d360e0034a40bb795c3c9161c4bc P 0f2814848fd04b67adbf350a3971f6ee: Deleting consensus metadata
I20260811 12:16:20.910251 20364 log.cc:1196] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/c084d360e0034a40bb795c3c9161c4bc
I20260811 12:16:20.910756 20364 ts_tablet_manager.cc:1939] T c084d360e0034a40bb795c3c9161c4bc P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting consensus metadata
I20260811 12:16:20.912144 19443 catalog_manager.cc:4841] TS 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927): tablet c084d360e0034a40bb795c3c9161c4bc (table test-table [id=e0b1c87843c44128a90f1c7592b61ad2]) successfully deleted
I20260811 12:16:20.913259 19438 catalog_manager.cc:4841] TS 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457): tablet c084d360e0034a40bb795c3c9161c4bc (table test-table [id=e0b1c87843c44128a90f1c7592b61ad2]) successfully deleted
Aug 11 12:16:20 dist-test-slave-rmhg krb5kdc[19395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:16:21 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:21 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:16:21.018597 20373 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:20.982440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.7.39.62:43987)
0811 12:16:20.998692 (+ 16252us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:20.998699 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:20.998728 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:16:20.998806 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:20.998812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:20.998912 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:20.999086 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:21.000633 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.001895 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.003876 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.004611 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.005477 (+ 866us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:21.018260 (+ 12783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53090: BlockingRecv error: recv got EOF from 127.0.0.1:53090 (error 108)
Metrics: {"server-negotiator.queue_time_us":16080,"thread_start_us":16788,"threads_started":1}
Aug 11 12:16:21 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM
I20260811 12:16:21.051983 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:21.031383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.7.39.62:43987)
0811 12:16:21.031584 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:21.031591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:21.032587 (+ 996us) server_negotiation.cc:407] Connection header received
0811 12:16:21.032773 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:21.032779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:21.032868 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:21.032973 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:21.034919 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.036090 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.038159 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.038844 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.038951 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:21.044507 (+ 5556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:21.044516 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:21.044521 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:21.044572 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:21.048369 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:21.049418 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:21.049425 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:21.049429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:21.049612 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:21.050130 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:21.050136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:21.050139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:21.050508 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:21.050773 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:21.051360 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 12:16:21.051618 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:16:21.057412 19460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:16:21.058210 19460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:16:21.067920 19878 tablet_service.cc:1467] Processing CreateTablet for tablet f4d8ef9f454e41a8adb1e498e3b742ed (DEFAULT_TABLE table=test-table [id=e84ac0d749fd4c7dbe76b65b94d7f166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:21.068084 19670 tablet_service.cc:1467] Processing CreateTablet for tablet f4d8ef9f454e41a8adb1e498e3b742ed (DEFAULT_TABLE table=test-table [id=e84ac0d749fd4c7dbe76b65b94d7f166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:21.068459 19878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d8ef9f454e41a8adb1e498e3b742ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:21.068593 19670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d8ef9f454e41a8adb1e498e3b742ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:21.069970 20101 tablet_service.cc:1467] Processing CreateTablet for tablet f4d8ef9f454e41a8adb1e498e3b742ed (DEFAULT_TABLE table=test-table [id=e84ac0d749fd4c7dbe76b65b94d7f166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:21.070400 20101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d8ef9f454e41a8adb1e498e3b742ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:21.082489 20384 tablet_bootstrap.cc:492] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: Bootstrap starting.
I20260811 12:16:21.085216 20384 tablet_bootstrap.cc:654] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:21.088649 20385 tablet_bootstrap.cc:492] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: Bootstrap starting.
I20260811 12:16:21.092839 20385 tablet_bootstrap.cc:654] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:21.097779 20387 tablet_bootstrap.cc:492] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: Bootstrap starting.
I20260811 12:16:21.100385 20387 tablet_bootstrap.cc:654] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:21.103159 20384 tablet_bootstrap.cc:492] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: No bootstrap required, opened a new log
I20260811 12:16:21.103415 20384 ts_tablet_manager.cc:1397] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.012s
I20260811 12:16:21.104390 20385 tablet_bootstrap.cc:492] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: No bootstrap required, opened a new log
I20260811 12:16:21.104293 20384 raft_consensus.cc:348] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.104579 20384 raft_consensus.cc:374] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:21.104676 20384 raft_consensus.cc:729] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2814848fd04b67adbf350a3971f6ee, State: Initialized, Role: FOLLOWER
I20260811 12:16:21.104925 20385 ts_tablet_manager.cc:1397] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
I20260811 12:16:21.104924 20384 consensus_queue.cc:260] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.106099 20385 raft_consensus.cc:348] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.106426 20385 raft_consensus.cc:374] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:21.106570 20385 raft_consensus.cc:729] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4794be8361c24aa6860e02cd3e520e4d, State: Initialized, Role: FOLLOWER
I20260811 12:16:21.106920 20385 consensus_queue.cc:260] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.108238 20384 ts_tablet_manager.cc:1428] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 12:16:21.108479 20385 ts_tablet_manager.cc:1428] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 12:16:21.116061 20387 tablet_bootstrap.cc:492] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: No bootstrap required, opened a new log
I20260811 12:16:21.116386 20387 ts_tablet_manager.cc:1397] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260811 12:16:21.117475 20387 raft_consensus.cc:348] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.117895 20387 raft_consensus.cc:374] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:21.118045 20387 raft_consensus.cc:729] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb5b83b29e4c76bc4b1a783ac1b19b, State: Initialized, Role: FOLLOWER
I20260811 12:16:21.119201 20387 consensus_queue.cc:260] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.121126 20387 ts_tablet_manager.cc:1428] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 12:16:21.151492 20359 raft_consensus.cc:482] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:21.151764 20359 raft_consensus.cc:504] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.152612 20359 leader_election.cc:288] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457), 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:21.153760 19713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d8ef9f454e41a8adb1e498e3b742ed" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" is_pre_election: true
I20260811 12:16:21.154026 19713 raft_consensus.cc:2454] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 0.
I20260811 12:16:21.154491 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d8ef9f454e41a8adb1e498e3b742ed" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d" is_pre_election: true
I20260811 12:16:21.154721 19910 raft_consensus.cc:2454] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 0.
I20260811 12:16:21.155457 20019 leader_election.cc:302] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 4794be8361c24aa6860e02cd3e520e4d; no voters:
I20260811 12:16:21.156684 20359 raft_consensus.cc:2781] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:21.156803 20359 raft_consensus.cc:482] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:21.156879 20359 raft_consensus.cc:3037] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:21.160370 20359 raft_consensus.cc:504] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.161970 19713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d8ef9f454e41a8adb1e498e3b742ed" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b"
I20260811 12:16:21.162201 19713 raft_consensus.cc:3037] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:21.165347 19713 raft_consensus.cc:2454] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 1.
I20260811 12:16:21.165935 20013 leader_election.cc:302] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 51bb5b83b29e4c76bc4b1a783ac1b19b; no voters:
I20260811 12:16:21.166296 20259 raft_consensus.cc:2781] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:21.166445 20259 raft_consensus.cc:686] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 1 LEADER]: Becoming Leader. State: Replica: 0f2814848fd04b67adbf350a3971f6ee, State: Running, Role: LEADER
I20260811 12:16:21.166708 20259 consensus_queue.cc:237] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.169929 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d8ef9f454e41a8adb1e498e3b742ed" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d"
I20260811 12:16:21.170169 19910 raft_consensus.cc:3037] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:21.175396 19910 raft_consensus.cc:2454] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 1.
I20260811 12:16:21.176272 20359 leader_election.cc:288] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 election: Requested vote from peers 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457), 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:21.179852 19463 catalog_manager.cc:5495] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3). New cstate: current_term: 1 leader_uuid: "0f2814848fd04b67adbf350a3971f6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } health_report { overall_health: HEALTHY } } }
I20260811 12:16:21.205672 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:21.200173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.7.39.3:32927)
0811 12:16:21.200385 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:21.200392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:21.200481 (+ 89us) server_negotiation.cc:407] Connection header received
0811 12:16:21.200716 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:21.200721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:21.200843 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:21.200937 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:21.201418 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.202246 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.203688 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.204187 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.204332 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:21.204809 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:21.204942 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:21.205027 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:21.205243 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 12:16:21.205337 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:16:21.208693 19713 raft_consensus.cc:1261] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Refusing update from remote peer 0f2814848fd04b67adbf350a3971f6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:21.208721 19910 raft_consensus.cc:1261] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Refusing update from remote peer 0f2814848fd04b67adbf350a3971f6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:21.209410 20359 consensus_queue.cc:1035] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:21.209702 20259 consensus_queue.cc:1035] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:21.225215 19460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:21.225468 19460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:21.228494 19460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: Sending DeleteTablet for 3 replicas of tablet f4d8ef9f454e41a8adb1e498e3b742ed
I20260811 12:16:21.229493 19670 tablet_service.cc:1514] Processing DeleteTablet for tablet f4d8ef9f454e41a8adb1e498e3b742ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:21 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:60280
I20260811 12:16:21.229624 19878 tablet_service.cc:1514] Processing DeleteTablet for tablet f4d8ef9f454e41a8adb1e498e3b742ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:21 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:60994
I20260811 12:16:21.229823 20364 tablet_replica.cc:330] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: stopping tablet replica
I20260811 12:16:21.229983 20364 raft_consensus.cc:2229] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:21.230185 20364 raft_consensus.cc:2258] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:21.230435 20362 tablet_replica.cc:330] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: stopping tablet replica
I20260811 12:16:21.230640 20362 raft_consensus.cc:2229] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:21.230885 20362 raft_consensus.cc:2258] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:21.231000 20101 tablet_service.cc:1514] Processing DeleteTablet for tablet f4d8ef9f454e41a8adb1e498e3b742ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:21 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:55964
I20260811 12:16:21.231078 19460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 12:16:21.231376 20363 tablet_replica.cc:330] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: stopping tablet replica
I20260811 12:16:21.231595 20363 raft_consensus.cc:2229] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 1 LEADER]: Raft consensus shutting down.
W20260811 12:16:21.231802 19460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:16:21.231967 20363 raft_consensus.cc:2258] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:21.233584 20364 ts_tablet_manager.cc:1905] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:21.239743 19670 tablet_service.cc:1467] Processing CreateTablet for tablet 36882ecc7b2f472280e267e7d0d07089 (DEFAULT_TABLE table=test-table [id=cec8e0b676c04e10ae1d739be06e2ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:21.240411 20101 tablet_service.cc:1467] Processing CreateTablet for tablet 36882ecc7b2f472280e267e7d0d07089 (DEFAULT_TABLE table=test-table [id=cec8e0b676c04e10ae1d739be06e2ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:21.240785 20362 ts_tablet_manager.cc:1905] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:21.240876 20101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36882ecc7b2f472280e267e7d0d07089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:21.245383 20362 ts_tablet_manager.cc:1918] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:21.245550 20362 log.cc:1196] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/f4d8ef9f454e41a8adb1e498e3b742ed
I20260811 12:16:21.245750 19670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36882ecc7b2f472280e267e7d0d07089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:21.246760 20364 ts_tablet_manager.cc:1918] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:21.247212 20363 ts_tablet_manager.cc:1905] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:21.248822 20364 log.cc:1196] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/f4d8ef9f454e41a8adb1e498e3b742ed
I20260811 12:16:21.246348 19878 tablet_service.cc:1467] Processing CreateTablet for tablet 36882ecc7b2f472280e267e7d0d07089 (DEFAULT_TABLE table=test-table [id=cec8e0b676c04e10ae1d739be06e2ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:21.249321 19878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36882ecc7b2f472280e267e7d0d07089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:21.249734 20364 ts_tablet_manager.cc:1939] T f4d8ef9f454e41a8adb1e498e3b742ed P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting consensus metadata
I20260811 12:16:21.250988 19438 catalog_manager.cc:4841] TS 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457): tablet f4d8ef9f454e41a8adb1e498e3b742ed (table test-table [id=e84ac0d749fd4c7dbe76b65b94d7f166]) successfully deleted
I20260811 12:16:21.251398 20384 tablet_bootstrap.cc:492] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: Bootstrap starting.
I20260811 12:16:21.252291 20362 ts_tablet_manager.cc:1939] T f4d8ef9f454e41a8adb1e498e3b742ed P 4794be8361c24aa6860e02cd3e520e4d: Deleting consensus metadata
I20260811 12:16:21.252496 20387 tablet_bootstrap.cc:492] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Bootstrap starting.
I20260811 12:16:21.254473 20384 tablet_bootstrap.cc:654] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:21.254828 20387 tablet_bootstrap.cc:654] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:21.255587 20385 tablet_bootstrap.cc:492] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: Bootstrap starting.
I20260811 12:16:21.256494 19442 catalog_manager.cc:4841] TS 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753): tablet f4d8ef9f454e41a8adb1e498e3b742ed (table test-table [id=e84ac0d749fd4c7dbe76b65b94d7f166]) successfully deleted
I20260811 12:16:21.257221 20385 tablet_bootstrap.cc:654] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:21.257959 20363 ts_tablet_manager.cc:1918] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:21.258132 20363 log.cc:1196] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/f4d8ef9f454e41a8adb1e498e3b742ed
I20260811 12:16:21.258591 20363 ts_tablet_manager.cc:1939] T f4d8ef9f454e41a8adb1e498e3b742ed P 0f2814848fd04b67adbf350a3971f6ee: Deleting consensus metadata
I20260811 12:16:21.259910 19443 catalog_manager.cc:4841] TS 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927): tablet f4d8ef9f454e41a8adb1e498e3b742ed (table test-table [id=e84ac0d749fd4c7dbe76b65b94d7f166]) successfully deleted
I20260811 12:16:21.264701 20387 tablet_bootstrap.cc:492] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: No bootstrap required, opened a new log
I20260811 12:16:21.264710 20384 tablet_bootstrap.cc:492] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: No bootstrap required, opened a new log
I20260811 12:16:21.264899 20387 ts_tablet_manager.cc:1397] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260811 12:16:21.264909 20384 ts_tablet_manager.cc:1397] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.008s
I20260811 12:16:21.265769 20384 raft_consensus.cc:348] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.266077 20384 raft_consensus.cc:374] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:21.266000 20387 raft_consensus.cc:348] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.266160 20384 raft_consensus.cc:729] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2814848fd04b67adbf350a3971f6ee, State: Initialized, Role: FOLLOWER
I20260811 12:16:21.266247 20387 raft_consensus.cc:374] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:21.266319 20387 raft_consensus.cc:729] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51bb5b83b29e4c76bc4b1a783ac1b19b, State: Initialized, Role: FOLLOWER
I20260811 12:16:21.266419 20384 consensus_queue.cc:260] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.266546 20387 consensus_queue.cc:260] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.267063 20387 ts_tablet_manager.cc:1428] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:16:21.267073 20384 ts_tablet_manager.cc:1428] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 12:16:21.272773 20385 tablet_bootstrap.cc:492] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: No bootstrap required, opened a new log
I20260811 12:16:21.272955 20385 ts_tablet_manager.cc:1397] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.001s
I20260811 12:16:21.273708 20385 raft_consensus.cc:348] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.273927 20385 raft_consensus.cc:374] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:21.274004 20385 raft_consensus.cc:729] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4794be8361c24aa6860e02cd3e520e4d, State: Initialized, Role: FOLLOWER
I20260811 12:16:21.274241 20385 consensus_queue.cc:260] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.274712 20385 ts_tablet_manager.cc:1428] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 12:16:21.478032 20259 raft_consensus.cc:482] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:21.478257 20259 raft_consensus.cc:504] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.479046 20259 leader_election.cc:288] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457), 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:21.479713 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36882ecc7b2f472280e267e7d0d07089" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d" is_pre_election: true
I20260811 12:16:21.479712 19713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36882ecc7b2f472280e267e7d0d07089" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" is_pre_election: true
I20260811 12:16:21.479987 19713 raft_consensus.cc:2454] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 0.
I20260811 12:16:21.479986 19910 raft_consensus.cc:2454] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 0.
I20260811 12:16:21.480461 20019 leader_election.cc:302] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 4794be8361c24aa6860e02cd3e520e4d; no voters:
I20260811 12:16:21.480751 20259 raft_consensus.cc:2781] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:21.480876 20259 raft_consensus.cc:482] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:21.480971 20259 raft_consensus.cc:3037] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:21.483124 20259 raft_consensus.cc:504] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.483691 20259 leader_election.cc:288] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 election: Requested vote from peers 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457), 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753)
I20260811 12:16:21.484284 19713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36882ecc7b2f472280e267e7d0d07089" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b"
I20260811 12:16:21.484309 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36882ecc7b2f472280e267e7d0d07089" candidate_uuid: "0f2814848fd04b67adbf350a3971f6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4794be8361c24aa6860e02cd3e520e4d"
I20260811 12:16:21.484480 19713 raft_consensus.cc:3037] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:21.484530 19910 raft_consensus.cc:3037] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:21.486547 19713 raft_consensus.cc:2454] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 1.
I20260811 12:16:21.486649 19910 raft_consensus.cc:2454] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f2814848fd04b67adbf350a3971f6ee in term 1.
I20260811 12:16:21.487027 20013 leader_election.cc:302] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2814848fd04b67adbf350a3971f6ee, 51bb5b83b29e4c76bc4b1a783ac1b19b; no voters:
I20260811 12:16:21.487406 20259 raft_consensus.cc:2781] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:21.487619 20259 raft_consensus.cc:686] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 1 LEADER]: Becoming Leader. State: Replica: 0f2814848fd04b67adbf350a3971f6ee, State: Running, Role: LEADER
I20260811 12:16:21.487955 20259 consensus_queue.cc:237] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } }
I20260811 12:16:21.490540 19463 catalog_manager.cc:5495] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3). New cstate: current_term: 1 leader_uuid: "0f2814848fd04b67adbf350a3971f6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f2814848fd04b67adbf350a3971f6ee" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32927 } health_report { overall_health: HEALTHY } } }
I20260811 12:16:21.521333 19910 raft_consensus.cc:1261] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Refusing update from remote peer 0f2814848fd04b67adbf350a3971f6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:21.521333 19713 raft_consensus.cc:1261] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Refusing update from remote peer 0f2814848fd04b67adbf350a3971f6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:21.521847 20359 consensus_queue.cc:1035] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "51bb5b83b29e4c76bc4b1a783ac1b19b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 37457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:21.522096 20259 consensus_queue.cc:1035] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "4794be8361c24aa6860e02cd3e520e4d" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:21.565506 19460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:21.565739 19460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:21.568655 19460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 475e0bae66f44457836d7734e1caa58d: Sending DeleteTablet for 3 replicas of tablet 36882ecc7b2f472280e267e7d0d07089
I20260811 12:16:21.569487 19878 tablet_service.cc:1514] Processing DeleteTablet for tablet 36882ecc7b2f472280e267e7d0d07089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:21 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:60994
I20260811 12:16:21.569514 19670 tablet_service.cc:1514] Processing DeleteTablet for tablet 36882ecc7b2f472280e267e7d0d07089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:21 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:60280
I20260811 12:16:21.569522 20101 tablet_service.cc:1514] Processing DeleteTablet for tablet 36882ecc7b2f472280e267e7d0d07089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:21 UTC) from {username='oryx', principal='oryx/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:55964
I20260811 12:16:21.569810 20363 tablet_replica.cc:330] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: stopping tablet replica
I20260811 12:16:21.569818 20364 tablet_replica.cc:330] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: stopping tablet replica
I20260811 12:16:21.570003 20364 raft_consensus.cc:2229] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:21.570003 20363 raft_consensus.cc:2229] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:16:21.570226 20364 raft_consensus.cc:2258] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:21.570356 20363 raft_consensus.cc:2258] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:21.570583 20362 tablet_replica.cc:330] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: stopping tablet replica
I20260811 12:16:21.570765 20362 raft_consensus.cc:2229] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:21.571025 20362 raft_consensus.cc:2258] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:21.571481 20363 ts_tablet_manager.cc:1905] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:21.571509 20364 ts_tablet_manager.cc:1905] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:21.572070 20362 ts_tablet_manager.cc:1905] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:21.576212 20363 ts_tablet_manager.cc:1918] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:16:21.576313 20363 log.cc:1196] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/36882ecc7b2f472280e267e7d0d07089
I20260811 12:16:21.576625 20363 ts_tablet_manager.cc:1939] T 36882ecc7b2f472280e267e7d0d07089 P 0f2814848fd04b67adbf350a3971f6ee: Deleting consensus metadata
I20260811 12:16:21.577860 19443 catalog_manager.cc:4841] TS 0f2814848fd04b67adbf350a3971f6ee (127.7.39.3:32927): tablet 36882ecc7b2f472280e267e7d0d07089 (table test-table [id=cec8e0b676c04e10ae1d739be06e2ec7]) successfully deleted
I20260811 12:16:21.578460 20362 ts_tablet_manager.cc:1918] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:16:21.578608 20362 log.cc:1196] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/36882ecc7b2f472280e267e7d0d07089
I20260811 12:16:21.578902 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 19573
I20260811 12:16:21.579056 20362 ts_tablet_manager.cc:1939] T 36882ecc7b2f472280e267e7d0d07089 P 4794be8361c24aa6860e02cd3e520e4d: Deleting consensus metadata
I20260811 12:16:21.579216 20364 ts_tablet_manager.cc:1918] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:16:21.579444 20364 log.cc:1196] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/36882ecc7b2f472280e267e7d0d07089
I20260811 12:16:21.579939 20364 ts_tablet_manager.cc:1939] T 36882ecc7b2f472280e267e7d0d07089 P 51bb5b83b29e4c76bc4b1a783ac1b19b: Deleting consensus metadata
I20260811 12:16:21.580838 19442 catalog_manager.cc:4841] TS 4794be8361c24aa6860e02cd3e520e4d (127.7.39.2:36753): tablet 36882ecc7b2f472280e267e7d0d07089 (table test-table [id=cec8e0b676c04e10ae1d739be06e2ec7]) successfully deleted
I20260811 12:16:21.581274 19438 catalog_manager.cc:4841] TS 51bb5b83b29e4c76bc4b1a783ac1b19b (127.7.39.1:37457): tablet 36882ecc7b2f472280e267e7d0d07089 (table test-table [id=cec8e0b676c04e10ae1d739be06e2ec7]) successfully deleted
Aug 11 12:16:21 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.1@KRBTEST.COM
I20260811 12:16:21.596374 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:21.580537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60312 (local address 127.7.39.1:37457)
0811 12:16:21.580994 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:21.580999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:21.581389 (+ 390us) server_negotiation.cc:407] Connection header received
0811 12:16:21.581613 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:21.581617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:21.581687 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:21.581782 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:21.582918 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.583617 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.584848 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.585384 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.585485 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:21.590096 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:21.590115 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:21.590138 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:21.590175 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:21.593354 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:21.594113 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:21.594122 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:21.594125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:21.594281 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:21.594674 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:21.594678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:21.594680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:21.595101 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:21.595429 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:21.595738 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 12:16:21.595980 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":224,"threads_started":1}
I20260811 12:16:21.597023 19775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:21.751314 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 19573
I20260811 12:16:21.779402 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 19788
Aug 11 12:16:21 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.2@KRBTEST.COM
I20260811 12:16:21.795372 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:21.780224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.7.39.2:36753)
0811 12:16:21.780734 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:21.780739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:21.780755 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:16:21.780808 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:21.780812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:21.780883 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:21.780990 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:21.782324 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.783072 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.784431 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.784864 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.784980 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:21.789386 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:21.789404 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:21.789423 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:21.789473 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:21.792449 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:21.793224 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:21.793235 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:21.793238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:21.793380 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:21.793778 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:21.793784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:21.793786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:21.794175 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:21.794463 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:21.794807 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 12:16:21.795035 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":237,"threads_started":1}
I20260811 12:16:21.796094 19973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:21.893685 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 19788
I20260811 12:16:21.920835 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 19984
Aug 11 12:16:21 dist-test-slave-rmhg krb5kdc[19395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.39.3@KRBTEST.COM
I20260811 12:16:21.938370 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:21.921629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.7.39.3:32927)
0811 12:16:21.922181 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:21.922188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:21.922207 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:16:21.922269 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:21.922274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:21.922353 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:21.922461 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:21.923834 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.925026 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.926351 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:21.926965 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:21.927099 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:21.931670 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:21.931694 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:21.931716 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:21.931773 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:21.935425 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:21.936160 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:21.936171 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:21.936175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:21.936350 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:21.936692 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:21.936697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:21.936701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:21.937146 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:21.937490 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:21.937719 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 12:16:21.937973 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":309,"threads_started":1}
I20260811 12:16:21.939154 20207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:22.064601 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 19984
I20260811 12:16:22.092695 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 19412
I20260811 12:16:22.093959 19520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:22.212446 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 19412
2026-08-11T12:16:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6270 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:16:22 dist-test-slave-rmhg krb5kdc[20429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:16:22 dist-test-slave-rmhg krb5kdc[20429](info): set up 2 sockets
Aug 11 12:16:22 dist-test-slave-rmhg krb5kdc[20429](info): commencing operation
krb5kdc: starting...
W20260811 12:16:23.475324 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.004s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:16:23 dist-test-slave-rmhg krb5kdc[20429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:16:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:16:23.761495 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:43453
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:44707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:43453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:24.106933 20458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:24.107244 20458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:24.107364 20458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:24.114743 20458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:24.114873 20458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:24.114928 20458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:24.115016 20458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:24.115084 20458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:24.125212 20458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:44707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:43453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:43453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:24.126582 20458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:24.127969 20458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:24.139902 20467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:24.139935 20463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:24.139969 20464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:24.140921 20458 server_base.cc:1034] running on GCE node
I20260811 12:16:24.141793 20458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:24.143309 20458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:24.144522 20458 hybrid_clock.cc:648] HybridClock initialized: now 1786450584144453 us; error 56 us; skew 500 ppm
I20260811 12:16:24.144882 20458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T12:16:24Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1998 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:16:24 dist-test-slave-rmhg krb5kdc[20478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:16:24 dist-test-slave-rmhg krb5kdc[20478](info): set up 2 sockets
Aug 11 12:16:24 dist-test-slave-rmhg krb5kdc[20478](info): commencing operation
krb5kdc: starting...
W20260811 12:16:25.467136 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:16:25 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:16:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:16:25.753299 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:44485
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:44485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:26.097950 20496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:26.098268 20496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:26.098388 20496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:26.105815 20496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:26.105950 20496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:26.105999 20496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:26.106078 20496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:26.106140 20496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:26.116345 20496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:44485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:44485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:26.117834 20496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:26.119210 20496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:26.131349 20503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:26.131395 20501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:26.131424 20505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:26.132238 20496 server_base.cc:1034] running on GCE node
I20260811 12:16:26.133105 20496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:26.134673 20496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:26.135896 20496 hybrid_clock.cc:648] HybridClock initialized: now 1786450586135843 us; error 46 us; skew 500 ppm
I20260811 12:16:26.136245 20496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:26 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450586, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:26.139417 20496 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:16:26.141043 20496 webserver.cc:460] Webserver started at http://127.7.39.62:45143/ using document root <none> and password file <none>
I20260811 12:16:26.141785 20496 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:26.141893 20496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:26.142179 20496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:26.145390 20496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "545b2f885c91466c88d93336a68f8b9d"
format_stamp: "Formatted at 2026-08-11 12:16:26 on dist-test-slave-rmhg"
server_key: "e34d64588565ab4d7ecf0b1db5af325e"
server_key_iv: "77148229a3029a113598bde7c86bed64"
server_key_version: "encryptionkey@0"
I20260811 12:16:26.146211 20496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "545b2f885c91466c88d93336a68f8b9d"
format_stamp: "Formatted at 2026-08-11 12:16:26 on dist-test-slave-rmhg"
server_key: "e34d64588565ab4d7ecf0b1db5af325e"
server_key_iv: "77148229a3029a113598bde7c86bed64"
server_key_version: "encryptionkey@0"
I20260811 12:16:26.151821 20496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:16:26.155690 20517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:26.157408 20496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:26.157599 20496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "545b2f885c91466c88d93336a68f8b9d"
format_stamp: "Formatted at 2026-08-11 12:16:26 on dist-test-slave-rmhg"
server_key: "e34d64588565ab4d7ecf0b1db5af325e"
server_key_iv: "77148229a3029a113598bde7c86bed64"
server_key_version: "encryptionkey@0"
I20260811 12:16:26.157800 20496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:26.182090 20496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:26.186877 20496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:26.187148 20496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:26.206079 20496 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:44485
I20260811 12:16:26.206127 20595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:44485 every 8 connection(s)
I20260811 12:16:26.207677 20496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:16:26.211977 20596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:26.211963 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 20496
I20260811 12:16:26.212188 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:16:26.212877 7324 external_mini_cluster.cc:1428] Setting key c9674e72af4f816754e521379f851874
I20260811 12:16:26.227015 20596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Bootstrap starting.
Aug 11 12:16:26 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:26.230384 20596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:26.231820 20596 log.cc:824] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:26.235496 20596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: No bootstrap required, opened a new log
I20260811 12:16:26.239166 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:26.215165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51084 (local address 127.7.39.62:44485)
0811 12:16:26.216059 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:26.216076 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:26.216133 (+ 57us) server_negotiation.cc:407] Connection header received
0811 12:16:26.218753 (+ 2620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:26.218790 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:26.219388 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:26.219894 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:26.221518 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:26.223611 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:26.224966 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:26.226092 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:26.226285 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:26.230270 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:26.230283 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:26.230306 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:26.230369 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:26.234125 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:26.234967 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:26.234977 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:26.234987 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:26.235218 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:26.235660 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:26.235665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:26.235668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:26.236434 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:26.236810 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:26.237101 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 12:16:26.237644 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":366,"threads_started":1}
I20260811 12:16:26.245014 20596 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } }
I20260811 12:16:26.245492 20596 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:26.245580 20596 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545b2f885c91466c88d93336a68f8b9d, State: Initialized, Role: FOLLOWER
I20260811 12:16:26.246179 20596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } }
I20260811 12:16:26.246407 20596 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:16:26.246510 20596 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:16:26.246646 20596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:26.249089 20596 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } }
I20260811 12:16:26.249666 20596 leader_election.cc:302] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545b2f885c91466c88d93336a68f8b9d; no voters:
I20260811 12:16:26.250247 20596 leader_election.cc:288] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:16:26.250399 20605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:26.251056 20605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 1 LEADER]: Becoming Leader. State: Replica: 545b2f885c91466c88d93336a68f8b9d, State: Running, Role: LEADER
I20260811 12:16:26.251641 20605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } }
I20260811 12:16:26.252383 20596 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:26.254395 20606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545b2f885c91466c88d93336a68f8b9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } } }
I20260811 12:16:26.254474 20607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545b2f885c91466c88d93336a68f8b9d. Latest consensus state: current_term: 1 leader_uuid: "545b2f885c91466c88d93336a68f8b9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } } }
I20260811 12:16:26.254849 20606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:26.255467 20607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:26.257694 20611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:26.263324 20611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:26.273474 20611 catalog_manager.cc:1347] Generated new cluster ID: 76cac7224ec54012b84750d3131b101b
I20260811 12:16:26.273610 20611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:26.287199 20611 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:16:26.288637 20611 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:26.296310 20611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Generated new TSK 0
I20260811 12:16:26.297333 20611 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:16:26.373651 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44485
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:26.722391 20631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:26.722692 20631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:26.722821 20631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:26.730252 20631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:26.730381 20631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:26.730544 20631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:26.740450 20631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:26.741904 20631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:26.743242 20631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:26.758050 20644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:26.758103 20641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:26.758029 20649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:26.759132 20631 server_base.cc:1034] running on GCE node
I20260811 12:16:26.759747 20631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:26.760733 20631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:26.761936 20631 hybrid_clock.cc:648] HybridClock initialized: now 1786450586761875 us; error 56 us; skew 500 ppm
I20260811 12:16:26.762279 20631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:26 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450586, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:26.765594 20631 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:16:26.767158 20631 webserver.cc:460] Webserver started at http://127.7.39.1:36937/ using document root <none> and password file <none>
I20260811 12:16:26.767879 20631 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:26.767997 20631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:26.768332 20631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:26.771531 20631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "939627c06e9f49a495d6e5102e6e8268"
format_stamp: "Formatted at 2026-08-11 12:16:26 on dist-test-slave-rmhg"
server_key: "a9edf3b1aec9e8be67582550086a43f6"
server_key_iv: "5fa65c2022f204bc316b4297daefcc0d"
server_key_version: "encryptionkey@0"
I20260811 12:16:26.772380 20631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "939627c06e9f49a495d6e5102e6e8268"
format_stamp: "Formatted at 2026-08-11 12:16:26 on dist-test-slave-rmhg"
server_key: "a9edf3b1aec9e8be67582550086a43f6"
server_key_iv: "5fa65c2022f204bc316b4297daefcc0d"
server_key_version: "encryptionkey@0"
I20260811 12:16:26.778213 20631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:16:26.782125 20660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:26.783715 20631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:26.783900 20631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "939627c06e9f49a495d6e5102e6e8268"
format_stamp: "Formatted at 2026-08-11 12:16:26 on dist-test-slave-rmhg"
server_key: "a9edf3b1aec9e8be67582550086a43f6"
server_key_iv: "5fa65c2022f204bc316b4297daefcc0d"
server_key_version: "encryptionkey@0"
I20260811 12:16:26.784126 20631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:26.796868 20631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:26.801560 20631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:26.801864 20631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:26.802906 20631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:26.804399 20631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:26.804500 20631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:26.804630 20631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:26.804685 20631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:26.836966 20631 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:46651
I20260811 12:16:26.837008 20819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:46651 every 8 connection(s)
I20260811 12:16:26.838325 20631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:26.843052 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 20631
I20260811 12:16:26.843245 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:16:26.843885 7324 external_mini_cluster.cc:1428] Setting key 83c7d99b84e3c2944d720f7a224069dc
Aug 11 12:16:26 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450586, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:26.858870 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:26.841048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:51345 (local address 127.7.39.62:44485)
0811 12:16:26.841535 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:26.841540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:26.842488 (+ 948us) server_negotiation.cc:407] Connection header received
0811 12:16:26.844442 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:26.844449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:26.844534 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:26.844634 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:26.846281 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:26.847021 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:26.848637 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:26.849047 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:26.849196 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:26.852913 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:26.852918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:26.852922 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:26.852957 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:26.855471 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:26.856295 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:26.856300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:26.856303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:26.856434 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:26.856914 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:26.856919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:26.856921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:26.857273 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:26.857414 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:26.858243 (+ 829us) server_negotiation.cc:299] Negotiation successful
0811 12:16:26.858490 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":265,"threads_started":1}
I20260811 12:16:26.859974 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:26.841567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:51345)
0811 12:16:26.842307 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:26.842358 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:26.844142 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:26.844902 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:26.844911 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:26.845493 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:26.846109 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:26.846128 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:26.847193 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:26.847198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:26.848484 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:26.848502 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:26.849318 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:26.849327 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:26.849564 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:26.850236 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:26.850259 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:26.852669 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:26.855647 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:26.855658 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:26.855676 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:26.856153 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:26.856552 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:26.856557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:26.856561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:26.856790 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:26.857403 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:26.857412 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:26.857799 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 12:16:26.858091 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 12:16:26.858687 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":291,"threads_started":1}
I20260811 12:16:26.861815 20820 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44485
I20260811 12:16:26.862149 20820 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:26.862942 20820 heartbeater.cc:511] Master 127.7.39.62:44485 requested a full tablet report, sending...
I20260811 12:16:26.865407 20546 ts_manager.cc:194] Registered new tserver with Master: 939627c06e9f49a495d6e5102e6e8268 (127.7.39.1:46651)
I20260811 12:16:26.867477 20546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:51345
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:16:26.908919 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44485
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:27.256079 20834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:27.256398 20834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:27.256527 20834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:27.264187 20834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:27.264338 20834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:27.264518 20834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:27.274552 20834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:27.276011 20834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:27.277436 20834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:27.290988 20844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:27.291076 20842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:27.291013 20841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:27.291872 20834 server_base.cc:1034] running on GCE node
I20260811 12:16:27.292510 20834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:27.293620 20834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:27.294833 20834 hybrid_clock.cc:648] HybridClock initialized: now 1786450587294775 us; error 50 us; skew 500 ppm
I20260811 12:16:27.295184 20834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:27 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450587, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:27.298651 20834 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:16:27.300305 20834 webserver.cc:460] Webserver started at http://127.7.39.2:33455/ using document root <none> and password file <none>
I20260811 12:16:27.301054 20834 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:27.301234 20834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:27.301579 20834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:27.304874 20834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "3d8e56620f2d4dd7844b39bc8927b81c"
format_stamp: "Formatted at 2026-08-11 12:16:27 on dist-test-slave-rmhg"
server_key: "89d09ab701232a054a3b850136981bad"
server_key_iv: "fba5e51ae371624fc0dbac5b49c3ddc7"
server_key_version: "encryptionkey@0"
I20260811 12:16:27.305836 20834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "3d8e56620f2d4dd7844b39bc8927b81c"
format_stamp: "Formatted at 2026-08-11 12:16:27 on dist-test-slave-rmhg"
server_key: "89d09ab701232a054a3b850136981bad"
server_key_iv: "fba5e51ae371624fc0dbac5b49c3ddc7"
server_key_version: "encryptionkey@0"
I20260811 12:16:27.311663 20834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:16:27.315709 20852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:27.317335 20834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:27.317569 20834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "3d8e56620f2d4dd7844b39bc8927b81c"
format_stamp: "Formatted at 2026-08-11 12:16:27 on dist-test-slave-rmhg"
server_key: "89d09ab701232a054a3b850136981bad"
server_key_iv: "fba5e51ae371624fc0dbac5b49c3ddc7"
server_key_version: "encryptionkey@0"
I20260811 12:16:27.317780 20834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:27.339766 20834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:27.344521 20834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:27.344806 20834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:27.345928 20834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:27.347508 20834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:27.347590 20834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:27.347678 20834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:27.347721 20834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:27.379552 20834 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:40183
I20260811 12:16:27.379598 21026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:40183 every 8 connection(s)
I20260811 12:16:27.380963 20834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:27.387306 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 20834
I20260811 12:16:27.387519 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:16:27.388113 7324 external_mini_cluster.cc:1428] Setting key a3fab09d2b09002f6011af2b1cb23187
Aug 11 12:16:27 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450587, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:27.403678 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:27.383861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:41897 (local address 127.7.39.62:44485)
0811 12:16:27.384379 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:27.384386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:27.385562 (+ 1176us) server_negotiation.cc:407] Connection header received
0811 12:16:27.387635 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:27.387642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:27.387740 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:27.387872 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:27.389812 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:27.390660 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:27.392925 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:27.393408 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:27.393554 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:27.397647 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:27.397652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:27.397656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:27.397692 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:27.400219 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:27.400977 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:27.400983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:27.400985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:27.401127 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:27.401654 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:27.401660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:27.401663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:27.402040 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:27.402190 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:27.403030 (+ 840us) server_negotiation.cc:299] Negotiation successful
0811 12:16:27.403301 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":251,"threads_started":1}
I20260811 12:16:27.404744 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:27.384517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:41897)
0811 12:16:27.385370 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:27.385427 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:27.387229 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:27.388267 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:27.388282 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:27.388886 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:27.389631 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:27.389656 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:27.390826 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:27.390832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:27.392754 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:27.392772 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:27.393680 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:27.393687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:27.393972 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:27.394740 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:27.394775 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:27.397382 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:27.400366 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:27.400376 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:27.400397 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:27.400840 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:27.401256 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:27.401262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:27.401265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:27.401515 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:27.402216 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:27.402223 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:27.402601 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 12:16:27.402886 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 12:16:27.403469 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":302,"threads_started":1}
I20260811 12:16:27.406453 21027 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44485
I20260811 12:16:27.406764 21027 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:27.407498 21027 heartbeater.cc:511] Master 127.7.39.62:44485 requested a full tablet report, sending...
I20260811 12:16:27.409116 20546 ts_manager.cc:194] Registered new tserver with Master: 3d8e56620f2d4dd7844b39bc8927b81c (127.7.39.2:40183)
I20260811 12:16:27.410234 20546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:41897
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:16:27.451822 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44485
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:27.799476 21034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:27.799798 21034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:27.799898 21034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:27.807395 21034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:27.807541 21034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:27.807667 21034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:27.817999 21034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:27.819356 21034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:27.820719 21034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:27.834163 21047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:27.834179 21048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:27.834297 21052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:27.834990 21034 server_base.cc:1034] running on GCE node
I20260811 12:16:27.835592 21034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:27.836655 21034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:27.837908 21034 hybrid_clock.cc:648] HybridClock initialized: now 1786450587837850 us; error 45 us; skew 500 ppm
I20260811 12:16:27.838281 21034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:27 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450587, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:27.842231 21034 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:16:27.843776 21034 webserver.cc:460] Webserver started at http://127.7.39.3:38949/ using document root <none> and password file <none>
I20260811 12:16:27.844462 21034 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:27.844578 21034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:27.844851 21034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:27.847918 21034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "2dc6eed5036e4c90bdeec3dcc26ffd24"
format_stamp: "Formatted at 2026-08-11 12:16:27 on dist-test-slave-rmhg"
server_key: "8e5f30c1ea7796a9e92382ae79c1faf2"
server_key_iv: "bcc5840ca7fd422e190b4041f0edcf70"
server_key_version: "encryptionkey@0"
I20260811 12:16:27.848713 21034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "2dc6eed5036e4c90bdeec3dcc26ffd24"
format_stamp: "Formatted at 2026-08-11 12:16:27 on dist-test-slave-rmhg"
server_key: "8e5f30c1ea7796a9e92382ae79c1faf2"
server_key_iv: "bcc5840ca7fd422e190b4041f0edcf70"
server_key_version: "encryptionkey@0"
I20260811 12:16:27.854269 21034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:16:27.858141 21062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:27.859692 21034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:27.859836 21034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "2dc6eed5036e4c90bdeec3dcc26ffd24"
format_stamp: "Formatted at 2026-08-11 12:16:27 on dist-test-slave-rmhg"
server_key: "8e5f30c1ea7796a9e92382ae79c1faf2"
server_key_iv: "bcc5840ca7fd422e190b4041f0edcf70"
server_key_version: "encryptionkey@0"
I20260811 12:16:27.860034 21034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:27.870764 20820 heartbeater.cc:503] Master 127.7.39.62:44485 was elected leader, sending a full tablet report...
I20260811 12:16:27.901055 21034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:27.905712 21034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:27.906028 21034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:27.907055 21034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:27.908493 21034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:27.908578 21034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:27.908679 21034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:27.908736 21034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:27.940035 21034 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:41513
I20260811 12:16:27.940133 21219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:41513 every 8 connection(s)
I20260811 12:16:27.941473 21034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:27.950745 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 21034
I20260811 12:16:27.950917 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:16:27.951371 7324 external_mini_cluster.cc:1428] Setting key a4751aebc05dbc83c309a88453ebd0d8
Aug 11 12:16:27 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450587, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:27.962014 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:27.944237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:52471 (local address 127.7.39.62:44485)
0811 12:16:27.944744 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:27.944749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:27.945744 (+ 995us) server_negotiation.cc:407] Connection header received
0811 12:16:27.947593 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:27.947599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:27.947681 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:27.947777 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:27.949366 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:27.950180 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:27.951822 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:27.952379 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:27.952497 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:27.956058 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:27.956064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:27.956067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:27.956101 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:27.958785 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:27.959555 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:27.959560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:27.959562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:27.959695 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:27.960130 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:27.960133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:27.960136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:27.960414 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:27.960552 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:27.961331 (+ 779us) server_negotiation.cc:299] Negotiation successful
0811 12:16:27.961606 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":245,"threads_started":1}
I20260811 12:16:27.962955 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:27.944823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:52471)
0811 12:16:27.945576 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:27.945623 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:27.947307 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:27.948028 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:27.948037 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:27.948572 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:27.949191 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:27.949216 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:27.950328 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:27.950334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:27.951679 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:27.951692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:27.952609 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:27.952616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:27.952866 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:27.953601 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:27.953623 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:27.955838 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:27.958935 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:27.958943 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:27.958960 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:27.959408 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:27.959804 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:27.959808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:27.959811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:27.960009 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:27.960554 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:27.960562 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:27.960915 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 12:16:27.961183 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 12:16:27.961784 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":313,"threads_started":1}
I20260811 12:16:27.964380 21220 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44485
I20260811 12:16:27.964738 21220 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:27.965653 21220 heartbeater.cc:511] Master 127.7.39.62:44485 requested a full tablet report, sending...
I20260811 12:16:27.967197 20546 ts_manager.cc:194] Registered new tserver with Master: 2dc6eed5036e4c90bdeec3dcc26ffd24 (127.7.39.3:41513)
I20260811 12:16:27.967964 20546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:52471
I20260811 12:16:27.979308 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab.
Entry for principal oryx/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab.
WARNING: no policy specified for oryx/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab.
Entry for principal oryx/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab.
WARNING: no policy specified for oryx/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab.
Entry for principal oryx/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab.
I20260811 12:16:28.152479 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 20631
Aug 11 12:16:28 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:16:28.169894 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:28.153683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.7.39.1:46651)
0811 12:16:28.154328 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:28.154335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:28.154390 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:16:28.154550 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:28.154558 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:28.154807 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:28.154955 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:28.156390 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:28.157571 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:28.158927 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:28.159566 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:28.159690 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:28.164086 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:28.164104 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:28.164125 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:28.164174 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:28.167200 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:28.167941 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:28.167948 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:28.167951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:28.168100 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:28.168400 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:28.168403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:28.168405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:28.168768 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:28.168998 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:28.169301 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 12:16:28.169519 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":303,"threads_started":1}
I20260811 12:16:28.171262 20813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:28.264685 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 20631
I20260811 12:16:28.297277 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 20834
Aug 11 12:16:28 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:16:28.315410 21243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:28.298300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38050 (local address 127.7.39.2:40183)
0811 12:16:28.298859 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:28.298867 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:28.298895 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:16:28.298959 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:28.298967 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:28.299186 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:28.299331 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:28.300693 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:28.301897 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:28.303329 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:28.303949 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:28.304065 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:28.308678 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:28.308697 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:28.308716 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:28.308773 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:28.311964 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:28.312852 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:28.312862 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:28.312864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:28.313041 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:28.313567 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:28.313572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:28.313574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:28.313995 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:28.314335 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:28.314738 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 12:16:28.315022 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":258,"threads_started":1}
I20260811 12:16:28.316784 21019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:28.405077 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 20834
I20260811 12:16:28.428138 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 21034
Aug 11 12:16:28 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:16:28.445279 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:28.429133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.7.39.3:41513)
0811 12:16:28.429701 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:28.429708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:28.429739 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:16:28.429800 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:28.429806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:28.430021 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:28.430167 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:28.431383 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:28.432377 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:28.433843 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:28.434465 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:28.434571 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:28.438856 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:28.438871 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:28.438890 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:28.438934 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:28.441987 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:28.442785 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:28.442793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:28.442795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:28.442965 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:28.443555 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:28.443560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:28.443564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:28.444038 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:28.444289 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:28.444628 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 12:16:28.444871 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":242,"threads_started":1}
I20260811 12:16:28.446637 21209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:28.529933 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 21034
I20260811 12:16:28.551883 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 20496
I20260811 12:16:28.553190 20591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:28.658212 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 20496
I20260811 12:16:28.688319 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:44485
--webserver_interface=127.7.39.62
--webserver_port=45143
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:44485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:29.031064 21251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:29.031373 21251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:29.031515 21251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:29.038795 21251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:29.038921 21251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:29.039000 21251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:29.039080 21251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:29.039139 21251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:29.049211 21251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:44485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:44485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=45143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:29.050611 21251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:29.051952 21251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:29.063536 21261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:29.063890 21259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:29.064155 21258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:29.064659 21251 server_base.cc:1034] running on GCE node
I20260811 12:16:29.065537 21251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:29.067030 21251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:29.068270 21251 hybrid_clock.cc:648] HybridClock initialized: now 1786450589068210 us; error 44 us; skew 500 ppm
I20260811 12:16:29.068638 21251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:29 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:29.071873 21251 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:16:29.073527 21251 webserver.cc:460] Webserver started at http://127.7.39.62:45143/ using document root <none> and password file <none>
I20260811 12:16:29.074287 21251 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:29.074424 21251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:29.080128 21251 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:16:29.083050 21271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:29.084471 21251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:16:29.084671 21251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "545b2f885c91466c88d93336a68f8b9d"
format_stamp: "Formatted at 2026-08-11 12:16:26 on dist-test-slave-rmhg"
server_key: "e34d64588565ab4d7ecf0b1db5af325e"
server_key_iv: "77148229a3029a113598bde7c86bed64"
server_key_version: "encryptionkey@0"
I20260811 12:16:29.085737 21251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:29.100587 21251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:29.104753 21251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:29.105010 21251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:29.123322 21251 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:44485
I20260811 12:16:29.123333 21358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:44485 every 8 connection(s)
I20260811 12:16:29.124922 21251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:16:29.132062 21359 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:16:29.135629 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 21251
I20260811 12:16:29.135771 21359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Bootstrap starting.
I20260811 12:16:29.136643 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:46651
--local_ip_for_outbound_sockets=127.7.39.1
--tserver_master_addrs=127.7.39.62:44485
--webserver_port=36937
--webserver_interface=127.7.39.1
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.39.1
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:16:29.157917 21359 log.cc:824] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:29.167410 21359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:16:29.168120 21359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Bootstrap complete.
I20260811 12:16:29.177637 21359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } }
I20260811 12:16:29.178573 21359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545b2f885c91466c88d93336a68f8b9d, State: Initialized, Role: FOLLOWER
I20260811 12:16:29.179242 21359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } }
I20260811 12:16:29.179483 21359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:16:29.179605 21359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:16:29.179901 21359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:16:29.183809 21359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } }
I20260811 12:16:29.184331 21359 leader_election.cc:302] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545b2f885c91466c88d93336a68f8b9d; no voters:
I20260811 12:16:29.184945 21359 leader_election.cc:288] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:16:29.185094 21367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:16:29.186237 21367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [term 2 LEADER]: Becoming Leader. State: Replica: 545b2f885c91466c88d93336a68f8b9d, State: Running, Role: LEADER
I20260811 12:16:29.186877 21367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } }
I20260811 12:16:29.187297 21359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:29.189558 21369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545b2f885c91466c88d93336a68f8b9d. Latest consensus state: current_term: 2 leader_uuid: "545b2f885c91466c88d93336a68f8b9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } } }
I20260811 12:16:29.189951 21369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:29.190202 21368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "545b2f885c91466c88d93336a68f8b9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545b2f885c91466c88d93336a68f8b9d" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44485 } } }
I20260811 12:16:29.190572 21368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:29.200639 21382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:29.206938 21382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:29.208539 21382 catalog_manager.cc:1259] Loaded cluster ID: 76cac7224ec54012b84750d3131b101b
I20260811 12:16:29.208731 21382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:29.212841 21382 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:29.213802 21392 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:16:29.214779 21382 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 545b2f885c91466c88d93336a68f8b9d: Loaded TSK: 0
I20260811 12:16:29.216425 21382 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:16:29.501305 21365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:29.501650 21365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:29.501801 21365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:29.509326 21365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:29.509446 21365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:29.509608 21365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:29.519785 21365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.1.keytab
--principal=oryx/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:46651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=36937
--tserver_master_addrs=127.7.39.62:44485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:29.521206 21365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:29.522535 21365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:29.537472 21399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:29.537506 21400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:29.538568 21402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:29.538686 21365 server_base.cc:1034] running on GCE node
I20260811 12:16:29.539453 21365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:29.540750 21365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:29.542008 21365 hybrid_clock.cc:648] HybridClock initialized: now 1786450589541930 us; error 56 us; skew 500 ppm
I20260811 12:16:29.542383 21365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:29 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450589, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:29.546430 21365 init.cc:377] Logged in from keytab as oryx/127.7.39.1@KRBTEST.COM (short username oryx)
I20260811 12:16:29.548203 21365 webserver.cc:460] Webserver started at http://127.7.39.1:36937/ using document root <none> and password file <none>
I20260811 12:16:29.549108 21365 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:29.549288 21365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:29.556028 21365 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:16:29.559118 21412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:29.560619 21365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:16:29.560771 21365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "939627c06e9f49a495d6e5102e6e8268"
format_stamp: "Formatted at 2026-08-11 12:16:26 on dist-test-slave-rmhg"
server_key: "a9edf3b1aec9e8be67582550086a43f6"
server_key_iv: "5fa65c2022f204bc316b4297daefcc0d"
server_key_version: "encryptionkey@0"
I20260811 12:16:29.561818 21365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:29.594636 21365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:29.599207 21365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:29.599465 21365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:29.600467 21365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:29.601950 21365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:29.602053 21365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:29.602139 21365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:29.602193 21365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:29.632864 21365 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:46651
I20260811 12:16:29.632922 21571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:46651 every 8 connection(s)
I20260811 12:16:29.634371 21365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:29.637853 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 21365
I20260811 12:16:29.638767 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:40183
--local_ip_for_outbound_sockets=127.7.39.2
--tserver_master_addrs=127.7.39.62:44485
--webserver_port=33455
--webserver_interface=127.7.39.2
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.39.2
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:16:29 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:29 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:29.653308 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:29.637705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:47139)
0811 12:16:29.638626 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:29.638673 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:29.640399 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:29.642394 (+ 1995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:29.642406 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:29.643028 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:29.643602 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:29.643622 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.645281 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.645286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:29.646526 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:29.646538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.647566 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.647574 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:29.647835 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:29.648501 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:29.648524 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:29.652226 (+ 3702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":378,"threads_started":1}
W20260811 12:16:29.653923 21575 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:29.637731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:47139 (local address 127.7.39.62:44485)
0811 12:16:29.641077 (+ 3346us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:29.641092 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:29.641144 (+ 52us) server_negotiation.cc:407] Connection header received
0811 12:16:29.641521 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:29.641534 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:29.641873 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:29.642208 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:29.643800 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.645085 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.646671 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.647326 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.647526 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:29.652702 (+ 5176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:47139: BlockingRecv error: recv got EOF from 127.7.39.1:47139 (error 108)
Metrics: {"server-negotiator.queue_time_us":3121,"thread_start_us":2913,"threads_started":1}
W20260811 12:16:29.654038 21572 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:44485 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.39.62:44485: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Aug 11 12:16:29 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:29 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:29.662761 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:29.654666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:57697)
0811 12:16:29.654847 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:29.654867 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:29.654971 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:29.655287 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:29.655290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:29.655589 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:29.655728 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:29.655735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.657063 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.657067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:29.657986 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:29.657998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.658807 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.658811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:29.658942 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:29.659669 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:29.659686 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:29.662558 (+ 2872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 12:16:29.663008 21575 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:29.654829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:57697 (local address 127.7.39.62:44485)
0811 12:16:29.654998 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:29.655003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:29.655021 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:16:29.655074 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:29.655079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:29.655162 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:29.655283 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:29.655858 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.656925 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.658138 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.658684 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.658797 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:29.662787 (+ 3990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:57697: BlockingRecv error: recv got EOF from 127.7.39.1:57697 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 12:16:29 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:29 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:29.671432 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:29.663414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:52445)
0811 12:16:29.663587 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:29.663607 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:29.663690 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:29.663969 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:29.663972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:29.664249 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:29.664386 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:29.664393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.665746 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.665750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:29.666801 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:29.666814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.667717 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.667722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:29.667846 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:29.668395 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:29.668411 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:29.671217 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 12:16:29.671731 21575 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:29.663490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:52445 (local address 127.7.39.62:44485)
0811 12:16:29.663654 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:29.663661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:29.663680 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:16:29.663773 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:29.663777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:29.663860 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:29.663962 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:29.664529 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.665610 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.666929 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:29.667584 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:29.667715 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:29.671480 (+ 3765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:52445: BlockingRecv error: recv got EOF from 127.7.39.1:52445 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 12:16:29.671733 21572 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:16:29.985651 21581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:29.985970 21581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:29.986080 21581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:29.993558 21581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:29.993678 21581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:29.993800 21581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:30.005535 21581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.2.keytab
--principal=oryx/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:40183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=33455
--tserver_master_addrs=127.7.39.62:44485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:30.009533 21581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:30.012899 21581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:30.026644 21591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:30.026682 21593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:30.026700 21590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:30.027523 21581 server_base.cc:1034] running on GCE node
I20260811 12:16:30.028133 21581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:30.029088 21581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:30.030315 21581 hybrid_clock.cc:648] HybridClock initialized: now 1786450590030254 us; error 50 us; skew 500 ppm
I20260811 12:16:30.030637 21581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450590, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:30.039477 21581 init.cc:377] Logged in from keytab as oryx/127.7.39.2@KRBTEST.COM (short username oryx)
I20260811 12:16:30.041205 21581 webserver.cc:460] Webserver started at http://127.7.39.2:33455/ using document root <none> and password file <none>
I20260811 12:16:30.041944 21581 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:30.042092 21581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:30.047744 21581 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 12:16:30.050529 21601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:30.051874 21581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:30.052028 21581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "3d8e56620f2d4dd7844b39bc8927b81c"
format_stamp: "Formatted at 2026-08-11 12:16:27 on dist-test-slave-rmhg"
server_key: "89d09ab701232a054a3b850136981bad"
server_key_iv: "fba5e51ae371624fc0dbac5b49c3ddc7"
server_key_version: "encryptionkey@0"
I20260811 12:16:30.053001 21581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:30.076222 21581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:30.080753 21581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:30.081033 21581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:30.082201 21581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:30.083621 21581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:30.083714 21581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:30.083828 21581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:30.083894 21581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:30.116166 21581 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:40183
I20260811 12:16:30.116214 21773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:40183 every 8 connection(s)
I20260811 12:16:30.117758 21581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:30.122006 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 21581
I20260811 12:16:30.122835 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:41513
--local_ip_for_outbound_sockets=127.7.39.3
--tserver_master_addrs=127.7.39.62:44485
--webserver_port=38949
--webserver_interface=127.7.39.3
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.39.3
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:16:30.137912 21575 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:30.120498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:40259 (local address 127.7.39.62:44485)
0811 12:16:30.120729 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:30.120736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:30.122130 (+ 1394us) server_negotiation.cc:407] Connection header received
0811 12:16:30.124733 (+ 2603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:30.124739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:30.124831 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:30.124945 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:30.127198 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.128360 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.130801 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.131470 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.131623 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.137661 (+ 6038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:40259: BlockingRecv error: recv got EOF from 127.7.39.2:40259 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:16:30.138733 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:30.121013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:40259)
0811 12:16:30.121901 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:30.121958 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:30.124340 (+ 2382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:30.125298 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:30.125310 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:30.126079 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:30.126922 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.126951 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.128598 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.128604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.130591 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.130613 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.131775 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.131783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.132104 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.132981 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:30.133027 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:30.137400 (+ 4373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":290,"threads_started":1}
W20260811 12:16:30.139492 21776 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:44485 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.39.62:44485: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:30.148386 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:30.140159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:46353)
0811 12:16:30.140347 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:30.140368 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:30.140489 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:30.140815 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:30.140819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:30.141208 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:30.141366 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.141375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.142425 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.142429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.143353 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.143366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.144112 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.144117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.144284 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.144947 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:30.144968 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:30.148085 (+ 3117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 12:16:30.148635 21575 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:30.140254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:46353 (local address 127.7.39.62:44485)
0811 12:16:30.140416 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:30.140422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:30.140441 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:16:30.140619 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:30.140623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:30.140687 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:30.140769 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:30.141526 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.142278 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.143496 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.143939 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.144066 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.148383 (+ 4317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:46353: BlockingRecv error: recv got EOF from 127.7.39.2:46353 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:30.158504 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:30.149351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:35585)
0811 12:16:30.149537 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:30.149557 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:30.149652 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:30.149982 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:30.149985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:30.150276 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:30.150434 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.150442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.151900 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.151904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.152863 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.152878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.153891 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.153896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.154041 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.154687 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:30.154706 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:30.158183 (+ 3477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:16:30.158811 21575 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:30.149476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:35585 (local address 127.7.39.62:44485)
0811 12:16:30.149664 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:30.149670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:30.149692 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:16:30.149762 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:30.149767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:30.149850 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:30.149971 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:30.150621 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.151737 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.153042 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.153714 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.153860 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.158412 (+ 4552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:35585: BlockingRecv error: recv got EOF from 127.7.39.2:35585 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 12:16:30.158831 21776 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:16:30.479398 21781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:30.479715 21781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:30.479851 21781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:30.487437 21781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:30.487571 21781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:30.487728 21781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:30.498059 21781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/krb5kdc/oryx_127.7.39.3.keytab
--principal=oryx/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:41513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=38949
--tserver_master_addrs=127.7.39.62:44485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:30.499464 21781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:30.500880 21781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:30.514853 21789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:30.514910 21795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:30.514822 21791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:30.515861 21781 server_base.cc:1034] running on GCE node
I20260811 12:16:30.516561 21781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:30.517712 21781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:30.518916 21781 hybrid_clock.cc:648] HybridClock initialized: now 1786450590518855 us; error 64 us; skew 500 ppm
I20260811 12:16:30.519248 21781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450590, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:30.522715 21781 init.cc:377] Logged in from keytab as oryx/127.7.39.3@KRBTEST.COM (short username oryx)
I20260811 12:16:30.524379 21781 webserver.cc:460] Webserver started at http://127.7.39.3:38949/ using document root <none> and password file <none>
I20260811 12:16:30.525135 21781 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:30.525339 21781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:30.531276 21781 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 12:16:30.534592 21805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:30.536319 21781 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 12:16:30.536517 21781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "2dc6eed5036e4c90bdeec3dcc26ffd24"
format_stamp: "Formatted at 2026-08-11 12:16:27 on dist-test-slave-rmhg"
server_key: "8e5f30c1ea7796a9e92382ae79c1faf2"
server_key_iv: "bcc5840ca7fd422e190b4041f0edcf70"
server_key_version: "encryptionkey@0"
I20260811 12:16:30.537659 21781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:30.553423 21781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:30.558151 21781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:30.558429 21781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:30.559417 21781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:30.560835 21781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:30.560926 21781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:30.561013 21781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:30.561070 21781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:30.591959 21781 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:41513
I20260811 12:16:30.592011 21972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:41513 every 8 connection(s)
I20260811 12:16:30.593569 21781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:30.595822 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 21781
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:16:30.610976 21575 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:30.596465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:37759 (local address 127.7.39.62:44485)
0811 12:16:30.596665 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:30.596672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:30.598364 (+ 1692us) server_negotiation.cc:407] Connection header received
0811 12:16:30.600282 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:30.600287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:30.600370 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:30.600469 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:30.602321 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.603142 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.604786 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.605289 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.605399 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.610698 (+ 5299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:37759: BlockingRecv error: recv got EOF from 127.7.39.3:37759 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:16:30.611698 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:30.597263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:37759)
0811 12:16:30.598177 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:30.598245 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:30.600007 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:30.600694 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:30.600703 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:30.601475 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:30.602133 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.602153 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.603302 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.603307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.604632 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.604646 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.605535 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.605543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.605776 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.606503 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:30.606526 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:30.610475 (+ 3949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":367,"threads_started":1}
W20260811 12:16:30.612447 21973 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:44485 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.39.62:44485: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
I20260811 12:16:30.612534 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:30.596842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48726 (local address 127.7.39.62:44485)
0811 12:16:30.598169 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:30.598174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:30.598190 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:16:30.598244 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:30.598247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:30.598307 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:30.598471 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:30.600052 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.600994 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.602357 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.602836 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.602950 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.605699 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:30.605709 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:30.605727 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:30.605771 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:30.608739 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:30.609698 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:30.609707 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:30.609714 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:30.609910 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:30.610333 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:30.610337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:30.610339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:30.610842 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:30.611302 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:30.611852 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 12:16:30.612247 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1229,"thread_start_us":1122,"threads_started":1}
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:30.621279 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:30.613088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:60377)
0811 12:16:30.613316 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:30.613338 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:30.613432 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:30.614261 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:30.614265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:30.614571 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:30.614748 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.614760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.615927 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.615931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.616906 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.616921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.617764 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.617769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.617901 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.618452 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:30.618470 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:30.621037 (+ 2567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101}
W20260811 12:16:30.621532 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:30.613771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:60377 (local address 127.7.39.62:44485)
0811 12:16:30.613940 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:30.613946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:30.613963 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:16:30.614010 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:30.614015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:30.614109 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:30.614219 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:30.614892 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.615788 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.617037 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.617624 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.617720 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.621306 (+ 3586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:60377: BlockingRecv error: recv got EOF from 127.7.39.3:60377 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:30.629671 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:30.622020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:43809)
0811 12:16:30.622177 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:30.622192 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:30.622286 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:30.622650 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:30.622654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:30.622940 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:30.623084 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.623092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.624050 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.624054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.625001 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.625013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.625708 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.625714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.625868 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.626356 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:30.626371 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:30.629449 (+ 3078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:16:30.629840 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:30.622137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:43809 (local address 127.7.39.62:44485)
0811 12:16:30.622405 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:30.622410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:30.622423 (+ 13us) server_negotiation.cc:407] Connection header received
0811 12:16:30.622459 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:30.622462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:30.622513 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:30.622597 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:30.623219 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.623912 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.625138 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.625574 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.625683 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.629636 (+ 3953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:43809: BlockingRecv error: recv got EOF from 127.7.39.3:43809 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
W20260811 12:16:30.629985 21973 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:30 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:30.681504 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:30.672501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:60861)
0811 12:16:30.673070 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:30.673088 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:30.673272 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:30.673630 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:30.673634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:30.674125 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:30.674327 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.674337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.675519 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.675531 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.676592 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:30.676607 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.677494 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.677500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:30.677659 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.678296 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:30.678316 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:30.681266 (+ 2950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":308,"threads_started":1}
W20260811 12:16:30.681677 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:30.672680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:60861 (local address 127.7.39.62:44485)
0811 12:16:30.672850 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:30.672854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:30.673235 (+ 381us) server_negotiation.cc:407] Connection header received
0811 12:16:30.673397 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:30.673401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:30.673473 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:30.673569 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:30.674536 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.675305 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.676751 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:30.677260 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:30.677372 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:30.681492 (+ 4120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:60861: BlockingRecv error: recv got EOF from 127.7.39.1:60861 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 12:16:31 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:31 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:31.168191 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:31.159468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:45043)
0811 12:16:31.160069 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:31.160087 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:31.160213 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:31.160529 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:31.160533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:31.161003 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:31.161238 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:31.161249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.162314 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.162319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:31.163327 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:31.163342 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.164074 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.164078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:31.164204 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:31.164847 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:31.164865 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:31.167929 (+ 3064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":338,"threads_started":1}
W20260811 12:16:31.168531 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:31.159582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:45043 (local address 127.7.39.62:44485)
0811 12:16:31.159751 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:31.159756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:31.160166 (+ 410us) server_negotiation.cc:407] Connection header received
0811 12:16:31.160310 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:31.160314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:31.160379 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:31.160460 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:31.161372 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.162141 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.163469 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.163908 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.164010 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:31.168260 (+ 4250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:45043: BlockingRecv error: recv got EOF from 127.7.39.2:45043 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:16:31 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:31 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:31.639897 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:31.630686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:46351)
0811 12:16:31.631185 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:31.631206 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:31.631335 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:31.631673 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:31.631676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:31.632167 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:31.632418 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:31.632431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.633569 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.633575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:31.635169 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:31.635187 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.635967 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.635972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:31.636114 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:31.636703 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:31.636722 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:31.639663 (+ 2941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":259,"threads_started":1}
W20260811 12:16:31.640132 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:31.630778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:46351 (local address 127.7.39.62:44485)
0811 12:16:31.631007 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:31.631014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:31.631290 (+ 276us) server_negotiation.cc:407] Connection header received
0811 12:16:31.631432 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:31.631436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:31.631502 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:31.631589 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:31.632597 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.633397 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.635350 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.635782 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.635880 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:31.639898 (+ 4018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:46351: BlockingRecv error: recv got EOF from 127.7.39.3:46351 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 12:16:31 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:31 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:31.691210 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:31.682496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:36973)
0811 12:16:31.683286 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:31.683304 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:31.683441 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:31.683774 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:31.683780 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:31.684277 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:31.684450 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:31.684461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.685550 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.685557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:31.686476 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:31.686489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.687184 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.687191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:31.687359 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:31.687975 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:31.687994 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:31.690953 (+ 2959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":508,"threads_started":1}
W20260811 12:16:31.691434 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:31.682603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:36973 (local address 127.7.39.62:44485)
0811 12:16:31.682846 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:31.682852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:31.683390 (+ 538us) server_negotiation.cc:407] Connection header received
0811 12:16:31.683565 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:31.683569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:31.683635 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:31.683724 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:31.684601 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.685409 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.686628 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:31.687031 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:31.687153 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:31.691179 (+ 4026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:36973: BlockingRecv error: recv got EOF from 127.7.39.1:36973 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 12:16:32 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:32 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:32.177450 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:32.169278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:35909)
0811 12:16:32.169872 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:32.169888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:32.170031 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:32.170449 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:32.170454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:32.170933 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:32.171088 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:32.171098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.172133 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.172137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:32.173031 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:32.173042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.173715 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.173719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:32.173829 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:32.174416 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:32.174435 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:32.177193 (+ 2758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":325,"threads_started":1}
W20260811 12:16:32.177625 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:32.169349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:35909 (local address 127.7.39.62:44485)
0811 12:16:32.169564 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:32.169571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:32.169982 (+ 411us) server_negotiation.cc:407] Connection header received
0811 12:16:32.170147 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:32.170152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:32.170294 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:32.170407 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:32.171232 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.172001 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.173180 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.173596 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.173689 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:32.177406 (+ 3717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:35909: BlockingRecv error: recv got EOF from 127.7.39.2:35909 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 12:16:32 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:32 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:32.650282 21989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:32.640938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:57859)
0811 12:16:32.641558 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:32.641585 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:32.641766 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:32.642233 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:32.642238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:32.642777 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:32.642991 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:32.643006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.644303 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.644308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:32.645471 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:32.645486 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.646203 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.646208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:32.646328 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:32.646972 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:32.646991 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:32.650024 (+ 3033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":342,"threads_started":1}
W20260811 12:16:32.650542 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:32.641200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:57859 (local address 127.7.39.62:44485)
0811 12:16:32.641427 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:32.641434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:32.641728 (+ 294us) server_negotiation.cc:407] Connection header received
0811 12:16:32.641897 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:32.641903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:32.642014 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:32.642143 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:32.643176 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.644137 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.645605 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.646038 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.646138 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:32.650280 (+ 4142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:57859: BlockingRecv error: recv got EOF from 127.7.39.3:57859 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 12:16:32 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:32 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:32.700894 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:32.692268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:33961)
0811 12:16:32.692765 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:32.692783 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:32.692920 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:32.693302 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:32.693307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:32.693690 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:32.693877 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:32.693886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.694876 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.694881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:32.695795 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:32.695809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.696545 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.696549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:32.696692 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:32.697317 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:32.697336 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:32.700604 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":271,"threads_started":1}
W20260811 12:16:32.701089 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:32.692481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:33961 (local address 127.7.39.62:44485)
0811 12:16:32.692672 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:32.692679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:32.692869 (+ 190us) server_negotiation.cc:407] Connection header received
0811 12:16:32.693041 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:32.693047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:32.693114 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:32.693232 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:32.694013 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.694742 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.695958 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:32.696379 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:32.696528 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:32.700866 (+ 4338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:33961: BlockingRecv error: recv got EOF from 127.7.39.1:33961 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 12:16:33 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:33 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:33.187007 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:33.178512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:34445)
0811 12:16:33.179008 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:33.179026 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:33.179147 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:33.179451 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:33.179454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:33.179810 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:33.180011 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:33.180021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.181047 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.181052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:33.181998 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:33.182012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.182774 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.182780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:33.182924 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:33.183590 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:33.183609 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:33.186691 (+ 3082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":256,"threads_started":1}
W20260811 12:16:33.187233 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:33.178543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:34445 (local address 127.7.39.62:44485)
0811 12:16:33.178729 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:33.178734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:33.179090 (+ 356us) server_negotiation.cc:407] Connection header received
0811 12:16:33.179236 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:33.179239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:33.179324 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:33.179409 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:33.180149 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.180904 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.182154 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.182608 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.182713 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:33.186976 (+ 4263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:34445: BlockingRecv error: recv got EOF from 127.7.39.2:34445 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 12:16:33 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:33 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:33.659566 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:33.651247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:55017)
0811 12:16:33.651766 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:33.651787 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:33.651924 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:33.652246 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:33.652250 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:33.652619 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:33.652794 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:33.652803 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.653927 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.653931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:33.654855 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:33.654868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.655590 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.655596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:33.655764 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:33.656330 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:33.656348 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:33.659230 (+ 2882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":296,"threads_started":1}
W20260811 12:16:33.659747 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:33.651356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:55017 (local address 127.7.39.62:44485)
0811 12:16:33.651567 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:33.651573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:33.651868 (+ 295us) server_negotiation.cc:407] Connection header received
0811 12:16:33.652038 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:33.652043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:33.652110 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:33.652193 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:33.652940 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.653792 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.655007 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.655439 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.655563 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:33.659466 (+ 3903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:55017: BlockingRecv error: recv got EOF from 127.7.39.3:55017 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 12:16:33 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:33 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:33.710806 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:33.701977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:32855)
0811 12:16:33.702569 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:33.702586 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:33.702728 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:33.703055 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:33.703060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:33.703474 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:33.703645 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:33.703655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.704708 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.704714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:33.705718 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:33.705732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.706437 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.706444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:33.706636 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:33.707242 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:33.707260 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:33.710487 (+ 3227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":324,"threads_started":1}
W20260811 12:16:33.710973 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:33.702096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:32855 (local address 127.7.39.62:44485)
0811 12:16:33.702346 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:33.702353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:33.702679 (+ 326us) server_negotiation.cc:407] Connection header received
0811 12:16:33.702849 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:33.702852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:33.702920 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:33.703015 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:33.703795 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.704574 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.705862 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:33.706294 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:33.706402 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:33.710729 (+ 4327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:32855: BlockingRecv error: recv got EOF from 127.7.39.1:32855 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 11 12:16:34 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:34 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:34.197431 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:34.188110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:51609)
0811 12:16:34.188720 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:34.188738 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:34.188879 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:34.189286 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:34.189314 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:34.189749 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:34.189932 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:34.189941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.191081 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.191088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:34.192490 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:34.192509 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.193357 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.193362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:34.193525 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:34.194155 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:34.194174 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:34.197129 (+ 2955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":270,"threads_started":1}
W20260811 12:16:34.197650 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:34.188171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:51609 (local address 127.7.39.62:44485)
0811 12:16:34.188374 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:34.188379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:34.188826 (+ 447us) server_negotiation.cc:407] Connection header received
0811 12:16:34.189007 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:34.189013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:34.189110 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:34.189232 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:34.190075 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.190886 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.192638 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.193181 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.193287 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:34.197408 (+ 4121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:51609: BlockingRecv error: recv got EOF from 127.7.39.2:51609 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 12:16:34 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:34 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:34.668936 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:34.660738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:47115)
0811 12:16:34.661323 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:34.661346 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:34.661468 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:34.661776 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:34.661779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:34.662146 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:34.662304 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:34.662312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.663308 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.663312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:34.664301 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:34.664314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.665014 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.665019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:34.665199 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:34.665815 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:34.665832 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:34.668678 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":314,"threads_started":1}
W20260811 12:16:34.669191 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:34.660846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:47115 (local address 127.7.39.62:44485)
0811 12:16:34.661036 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:34.661043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:34.661418 (+ 375us) server_negotiation.cc:407] Connection header received
0811 12:16:34.661581 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:34.661584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:34.661651 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:34.661738 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:34.662419 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.663177 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.664452 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.664877 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.664971 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:34.668890 (+ 3919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:47115: BlockingRecv error: recv got EOF from 127.7.39.3:47115 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 12:16:34 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:34 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:34.720580 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:34.711863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:45701)
0811 12:16:34.712429 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:34.712446 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:34.712584 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:34.712922 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:34.712927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:34.713424 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:34.713612 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:34.713621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.714722 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.714727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:34.715686 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:34.715700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.716490 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.716496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:34.716675 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:34.717297 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:34.717315 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:34.720335 (+ 3020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":274,"threads_started":1}
W20260811 12:16:34.720717 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:34.711995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:45701 (local address 127.7.39.62:44485)
0811 12:16:34.712233 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:34.712239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:34.712540 (+ 301us) server_negotiation.cc:407] Connection header received
0811 12:16:34.712712 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:34.712717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:34.712785 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:34.712874 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:34.713768 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.714573 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.715835 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:34.716331 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:34.716460 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:34.720529 (+ 4069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:45701: BlockingRecv error: recv got EOF from 127.7.39.1:45701 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 12:16:35 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:35 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:35.210842 22006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:35.198576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:34171)
0811 12:16:35.199124 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:35.199144 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:35.199306 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:35.199684 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:35.199689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:35.200114 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:35.200313 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:35.200323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.201508 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.201513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:35.202541 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:35.202560 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.203499 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.203506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:35.203699 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:35.207587 (+ 3888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:35.207608 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:35.210505 (+ 2897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":247,"threads_started":1}
W20260811 12:16:35.211068 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:35.198861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:34171 (local address 127.7.39.62:44485)
0811 12:16:35.199103 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:35.199110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:35.199246 (+ 136us) server_negotiation.cc:407] Connection header received
0811 12:16:35.199434 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:35.199439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:35.199531 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:35.199622 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:35.200479 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.201336 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.202751 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.203337 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.203444 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:35.210797 (+ 7353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:34171: BlockingRecv error: recv got EOF from 127.7.39.2:34171 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 12:16:35 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:35 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:35.678305 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:35.669909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:42269)
0811 12:16:35.670433 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:35.670453 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:35.670587 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:35.670921 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:35.670925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:35.671367 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:35.671541 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:35.671551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.672613 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.672618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:35.673633 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:35.673646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.674419 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.674424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:35.674590 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:35.675178 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:35.675195 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:35.677966 (+ 2771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":253,"threads_started":1}
W20260811 12:16:35.678435 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:35.670016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:42269 (local address 127.7.39.62:44485)
0811 12:16:35.670216 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:35.670223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:35.670568 (+ 345us) server_negotiation.cc:407] Connection header received
0811 12:16:35.670712 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:35.670716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:35.670782 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:35.670868 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:35.671712 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.672468 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.673788 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.674272 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.674391 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:35.678211 (+ 3820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:42269: BlockingRecv error: recv got EOF from 127.7.39.3:42269 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 12:16:35 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:35 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:35.730764 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:35.721653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:36295)
0811 12:16:35.722268 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:35.722287 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:35.722416 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:35.722905 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:35.722910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:35.723452 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:35.723656 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:35.723668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.724852 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.724858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:35.725846 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:35.725860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.726618 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.726624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:35.726805 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:35.727456 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:35.727475 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:35.730529 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":314,"threads_started":1}
W20260811 12:16:35.731030 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:35.721787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:36295 (local address 127.7.39.62:44485)
0811 12:16:35.721969 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:35.721975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:35.722393 (+ 418us) server_negotiation.cc:407] Connection header received
0811 12:16:35.722588 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:35.722594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:35.722686 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:35.722792 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:35.723810 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.724686 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.726001 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:35.726461 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:35.726576 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:35.730774 (+ 4198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:36295: BlockingRecv error: recv got EOF from 127.7.39.1:36295 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:16:36 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:36 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:36.220484 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:36.211897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:35241)
0811 12:16:36.212450 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:36.212466 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:36.212606 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:36.213003 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:36.213008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:36.213508 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:36.213688 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:36.213698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.214739 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.214743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:36.215654 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:36.215677 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.216459 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.216464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:36.216614 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:36.217229 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:36.217254 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:36.220197 (+ 2943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":324,"threads_started":1}
W20260811 12:16:36.220691 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:36.211990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:35241 (local address 127.7.39.62:44485)
0811 12:16:36.212207 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:36.212213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:36.212556 (+ 343us) server_negotiation.cc:407] Connection header received
0811 12:16:36.212770 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:36.212775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:36.212863 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:36.212949 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:36.213839 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.214591 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.215815 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.216257 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.216392 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:36.220462 (+ 4070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:35241: BlockingRecv error: recv got EOF from 127.7.39.2:35241 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 12:16:36 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:36 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:36.687492 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:36.679308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:35141)
0811 12:16:36.679926 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:36.679949 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:36.680068 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:36.680434 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:36.680437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:36.680873 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:36.681030 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:36.681039 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.682128 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.682132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:36.683057 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:36.683069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.683764 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.683769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:36.683895 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:36.684474 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:36.684491 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:36.687154 (+ 2663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":381,"threads_started":1}
W20260811 12:16:36.687736 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:36.679424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:35141 (local address 127.7.39.62:44485)
0811 12:16:36.679595 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:36.679600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:36.680017 (+ 417us) server_negotiation.cc:407] Connection header received
0811 12:16:36.680211 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:36.680216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:36.680302 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:36.680406 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:36.681206 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.681982 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.683190 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.683607 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.683715 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:36.687424 (+ 3709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:35141: BlockingRecv error: recv got EOF from 127.7.39.3:35141 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 12:16:36 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:36 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:36.740648 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:36.731815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:34081)
0811 12:16:36.732520 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:36.732538 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:36.732687 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:36.733031 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:36.733036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:36.733550 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:36.733729 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:36.733739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.734802 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.734808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:36.735736 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:36.735749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.736508 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.736514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:36.736669 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:36.737291 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:36.737312 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:36.740402 (+ 3090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":293,"threads_started":1}
W20260811 12:16:36.740847 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:36.731961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:34081 (local address 127.7.39.62:44485)
0811 12:16:36.732197 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:36.732204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:36.732637 (+ 433us) server_negotiation.cc:407] Connection header received
0811 12:16:36.732814 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:36.732818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:36.732883 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:36.732979 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:36.733903 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.734649 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.735886 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:36.736315 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:36.736435 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:36.740638 (+ 4203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:34081: BlockingRecv error: recv got EOF from 127.7.39.1:34081 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 12:16:37 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:37 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:37.230408 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:37.221693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:46673)
0811 12:16:37.222363 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:37.222385 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:37.222551 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:37.222948 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:37.222953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:37.223500 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:37.223695 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:37.223707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.224815 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.224819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:37.225831 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:37.225846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.226529 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.226534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:37.226650 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:37.227218 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:37.227236 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:37.230153 (+ 2917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":264,"threads_started":1}
W20260811 12:16:37.230594 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:37.221824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:46673 (local address 127.7.39.62:44485)
0811 12:16:37.222077 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:37.222082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:37.222500 (+ 418us) server_negotiation.cc:407] Connection header received
0811 12:16:37.222678 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:37.222683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:37.222795 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:37.222880 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:37.223855 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.224648 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.225955 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.226394 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.226490 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:37.230361 (+ 3871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:46673: BlockingRecv error: recv got EOF from 127.7.39.2:46673 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 11 12:16:37 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:37 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:37.697242 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:37.688457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:58381)
0811 12:16:37.689083 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:37.689105 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:37.689272 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:37.689551 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:37.689555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:37.690008 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:37.690227 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:37.690239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.691358 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.691363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:37.692360 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:37.692373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.693044 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.693048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:37.693192 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:37.693742 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:37.693759 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:37.696946 (+ 3187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":248,"threads_started":1}
W20260811 12:16:37.697398 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:37.688523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:58381 (local address 127.7.39.62:44485)
0811 12:16:37.688748 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:37.688756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:37.689191 (+ 435us) server_negotiation.cc:407] Connection header received
0811 12:16:37.689372 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:37.689375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:37.689442 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:37.689524 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:37.690364 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.691218 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.692488 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.692923 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.693017 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:37.697185 (+ 4168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:58381: BlockingRecv error: recv got EOF from 127.7.39.3:58381 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 12:16:37 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:37 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:37.750931 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:37.741643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:59359)
0811 12:16:37.742300 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:37.742317 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:37.742477 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:37.742848 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:37.742853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:37.743313 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:37.743484 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:37.743494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.744777 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.744784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:37.745932 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:37.745948 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.746735 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.746743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:37.746929 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:37.747553 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:37.747580 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:37.750682 (+ 3102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":367,"threads_started":1}
W20260811 12:16:37.751214 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:37.741930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:59359 (local address 127.7.39.62:44485)
0811 12:16:37.742230 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:37.742236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:37.742415 (+ 179us) server_negotiation.cc:407] Connection header received
0811 12:16:37.742618 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:37.742622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:37.742694 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:37.742780 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:37.743685 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.744621 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.746085 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:37.746535 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:37.746656 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:37.750967 (+ 4311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:59359: BlockingRecv error: recv got EOF from 127.7.39.1:59359 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 12:16:38 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:38 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:38.240716 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:38.231495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:39671)
0811 12:16:38.232070 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:38.232086 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:38.232234 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:38.232624 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:38.232628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:38.233072 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:38.233288 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:38.233299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.234524 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.234529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:38.235484 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:38.235497 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.236275 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.236281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:38.236458 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:38.237110 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:38.237127 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:38.240405 (+ 3278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":315,"threads_started":1}
W20260811 12:16:38.240988 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:38.231654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:39671 (local address 127.7.39.62:44485)
0811 12:16:38.231924 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:38.231930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:38.232192 (+ 262us) server_negotiation.cc:407] Connection header received
0811 12:16:38.232360 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:38.232365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:38.232466 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:38.232556 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:38.233457 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.234362 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.235644 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.236069 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.236199 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:38.240700 (+ 4501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:39671: BlockingRecv error: recv got EOF from 127.7.39.2:39671 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 12:16:38 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:38 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:38.706791 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:38.698314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:46311)
0811 12:16:38.698814 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:38.698834 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:38.698972 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:38.699304 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:38.699309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:38.699751 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:38.699914 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:38.699924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.700947 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.700952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:38.701925 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:38.701938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.702707 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.702714 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:38.702865 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:38.703486 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:38.703504 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:38.706565 (+ 3061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":263,"threads_started":1}
W20260811 12:16:38.707039 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:38.698407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:46311 (local address 127.7.39.62:44485)
0811 12:16:38.698595 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:38.698601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:38.698919 (+ 318us) server_negotiation.cc:407] Connection header received
0811 12:16:38.699087 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:38.699091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:38.699173 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:38.699256 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:38.700058 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.700806 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.702066 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.702512 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.702625 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:38.706777 (+ 4152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:46311: BlockingRecv error: recv got EOF from 127.7.39.3:46311 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 12:16:38 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:38 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:38.761230 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:38.752024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:50291)
0811 12:16:38.752909 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:38.752929 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:38.753081 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:38.753439 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:38.753442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:38.753984 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:38.754217 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:38.754232 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.755316 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.755320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:38.756293 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:38.756306 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.757086 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.757090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:38.757252 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:38.757886 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:38.757905 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:38.760925 (+ 3020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":621,"threads_started":1}
W20260811 12:16:38.761426 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:38.752176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:50291 (local address 127.7.39.62:44485)
0811 12:16:38.752364 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:38.752371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:38.753024 (+ 653us) server_negotiation.cc:407] Connection header received
0811 12:16:38.753216 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:38.753221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:38.753304 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:38.753409 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:38.754376 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.755176 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.756450 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:38.756935 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:38.757074 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:38.761221 (+ 4147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:50291: BlockingRecv error: recv got EOF from 127.7.39.1:50291 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:16:39 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:39 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:39.250571 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:39.241925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:51337)
0811 12:16:39.242525 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:39.242546 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:39.242701 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:39.243093 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:39.243100 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:39.243637 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:39.243862 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:39.243873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.244883 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.244887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:39.245899 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:39.245912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.246643 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.246647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:39.246789 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:39.247343 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:39.247360 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:39.250303 (+ 2943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":308,"threads_started":1}
W20260811 12:16:39.250933 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:39.242000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:51337 (local address 127.7.39.62:44485)
0811 12:16:39.242237 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:39.242245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:39.242631 (+ 386us) server_negotiation.cc:407] Connection header received
0811 12:16:39.242836 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:39.242841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:39.242926 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:39.243019 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:39.243993 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.244768 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.246038 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.246498 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.246605 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:39.250651 (+ 4046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:51337: BlockingRecv error: recv got EOF from 127.7.39.2:51337 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 12:16:39 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:39 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:39.715904 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:39.707639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:53427)
0811 12:16:39.708170 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:39.708190 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:39.708324 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:39.708643 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:39.708647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:39.709071 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:39.709273 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:39.709283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.710269 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.710273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:39.711231 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:39.711244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.711954 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.711960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:39.712134 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:39.712713 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:39.712732 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:39.715567 (+ 2835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":289,"threads_started":1}
W20260811 12:16:39.716025 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:39.707747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:53427 (local address 127.7.39.62:44485)
0811 12:16:39.707937 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:39.707943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:39.708275 (+ 332us) server_negotiation.cc:407] Connection header received
0811 12:16:39.708439 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:39.708444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:39.708507 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:39.708590 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:39.709421 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.710126 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.711376 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.711804 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.711914 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:39.715807 (+ 3893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:53427: BlockingRecv error: recv got EOF from 127.7.39.3:53427 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 12:16:39 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:39 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:39.771535 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:39.762233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:53281)
0811 12:16:39.762742 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:39.762758 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:39.762878 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:39.763189 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:39.763193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:39.763638 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:39.763862 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:39.763876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.765584 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.765591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:39.766889 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:39.766909 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.767645 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.767650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:39.767783 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:39.768395 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:39.768415 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:39.771277 (+ 2862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":274,"threads_started":1}
W20260811 12:16:39.771711 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:39.762363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:53281 (local address 127.7.39.62:44485)
0811 12:16:39.762530 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:39.762535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:39.762830 (+ 295us) server_negotiation.cc:407] Connection header received
0811 12:16:39.762973 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:39.762977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:39.763050 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:39.763143 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:39.764073 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.765171 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.767040 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:39.767480 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:39.767580 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:39.771486 (+ 3906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:53281: BlockingRecv error: recv got EOF from 127.7.39.1:53281 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 12:16:40 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:40 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:40.264954 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:40.251629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:48929)
0811 12:16:40.252208 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:40.252225 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:40.252363 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:40.252725 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:40.252730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:40.253201 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:40.253366 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:40.253376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.254456 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.254460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:40.255424 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:40.255438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.256232 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.256238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:40.256413 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:40.261234 (+ 4821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:40.261259 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:40.264693 (+ 3434us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":296,"threads_started":1}
W20260811 12:16:40.265128 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:40.251759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:48929 (local address 127.7.39.62:44485)
0811 12:16:40.251998 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:40.252006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:40.252314 (+ 308us) server_negotiation.cc:407] Connection header received
0811 12:16:40.252480 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:40.252483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:40.252581 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:40.252670 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:40.253519 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.254309 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.255581 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.256023 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.256185 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:40.264895 (+ 8710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:48929: BlockingRecv error: recv got EOF from 127.7.39.2:48929 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 12:16:40 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:40 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:40.725523 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:40.716932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:48013)
0811 12:16:40.717519 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:40.717540 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:40.717680 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:40.718043 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:40.718048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:40.718451 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:40.718640 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:40.718649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.719663 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.719669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:40.720624 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:40.720636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.721463 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.721469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:40.721661 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:40.722242 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:40.722259 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:40.725209 (+ 2950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":345,"threads_started":1}
W20260811 12:16:40.725823 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:40.717180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:48013 (local address 127.7.39.62:44485)
0811 12:16:40.717408 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:40.717415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:40.717626 (+ 211us) server_negotiation.cc:407] Connection header received
0811 12:16:40.717801 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:40.717807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:40.717898 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:40.717985 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:40.718801 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.719523 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.720766 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.721308 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.721423 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:40.725577 (+ 4154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:48013: BlockingRecv error: recv got EOF from 127.7.39.3:48013 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 12:16:40 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:40 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:40.781772 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:40.772585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:35473)
0811 12:16:40.773181 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:40.773203 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:40.773346 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:40.773700 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:40.773704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:40.774149 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:40.774341 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:40.774352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.775415 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.775421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:40.776349 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:40.776362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.777305 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.777312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:40.777497 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:40.778278 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:40.778304 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:40.781514 (+ 3210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":299,"threads_started":1}
W20260811 12:16:40.782044 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:40.772773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:35473 (local address 127.7.39.62:44485)
0811 12:16:40.773003 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:40.773010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:40.773283 (+ 273us) server_negotiation.cc:407] Connection header received
0811 12:16:40.773478 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:40.773482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:40.773552 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:40.773641 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:40.774502 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.775273 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.776500 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:40.776999 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:40.777177 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:40.781777 (+ 4600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:35473: BlockingRecv error: recv got EOF from 127.7.39.1:35473 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 12:16:41 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:41 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:41.274773 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:41.266128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:34963)
0811 12:16:41.266751 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:41.266770 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:41.266934 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:41.267311 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:41.267317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:41.267878 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:41.268077 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:41.268087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.269133 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.269139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:41.270107 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:41.270121 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.270830 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.270837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:41.271019 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:41.271587 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:41.271607 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:41.274429 (+ 2822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":341,"threads_started":1}
W20260811 12:16:41.274941 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:41.266283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:34963 (local address 127.7.39.62:44485)
0811 12:16:41.266506 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:41.266513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:41.266869 (+ 356us) server_negotiation.cc:407] Connection header received
0811 12:16:41.267068 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:41.267072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:41.267142 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:41.267244 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:41.268226 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.268992 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.270257 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.270679 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.270790 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:41.274679 (+ 3889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:34963: BlockingRecv error: recv got EOF from 127.7.39.2:34963 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 12:16:41 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:41 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:41.735737 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:41.726775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:54211)
0811 12:16:41.727337 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:41.727360 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:41.727492 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:41.727897 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:41.727900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:41.728335 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:41.728529 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:41.728540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.729900 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.729906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:41.731003 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:41.731019 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.731733 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.731737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:41.731857 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:41.732509 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:41.732527 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:41.735480 (+ 2953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":317,"threads_started":1}
W20260811 12:16:41.735957 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:41.726933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:54211 (local address 127.7.39.62:44485)
0811 12:16:41.727113 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:41.727118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:41.727435 (+ 317us) server_negotiation.cc:407] Connection header received
0811 12:16:41.727668 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:41.727673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:41.727745 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:41.727842 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:41.728716 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.729730 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.731140 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.731595 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.731707 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:41.735742 (+ 4035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:54211: BlockingRecv error: recv got EOF from 127.7.39.3:54211 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 12:16:41 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:41 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:41.791962 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:41.782832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:52155)
0811 12:16:41.783414 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:41.783431 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:41.783569 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:41.783917 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:41.783922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:41.784354 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:41.784544 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:41.784553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.785735 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.785739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:41.786719 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:41.786734 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.787554 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.787561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:41.787764 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:41.788481 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:41.788501 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:41.791557 (+ 3056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":319,"threads_started":1}
W20260811 12:16:41.792148 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:41.782926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:52155 (local address 127.7.39.62:44485)
0811 12:16:41.783137 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:41.783144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:41.783518 (+ 374us) server_negotiation.cc:407] Connection header received
0811 12:16:41.783690 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:41.783696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:41.783769 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:41.783854 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:41.784699 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.785578 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.786882 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:41.787380 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:41.787516 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:41.791808 (+ 4292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:52155: BlockingRecv error: recv got EOF from 127.7.39.1:52155 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 12:16:42 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:42 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:42.284904 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:42.275901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:51879)
0811 12:16:42.276480 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:42.276498 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:42.276642 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:42.277031 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:42.277036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:42.277521 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:42.277705 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:42.277716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.278924 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.278929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:42.280003 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:42.280017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.280761 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.280767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:42.280945 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:42.281622 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:42.281641 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:42.284567 (+ 2926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":313,"threads_started":1}
W20260811 12:16:42.285068 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:42.276013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:51879 (local address 127.7.39.62:44485)
0811 12:16:42.276313 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:42.276319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:42.276589 (+ 270us) server_negotiation.cc:407] Connection header received
0811 12:16:42.276803 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:42.276808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:42.276876 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:42.276965 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:42.277865 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.278764 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.280158 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.280605 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.280720 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:42.284802 (+ 4082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:51879: BlockingRecv error: recv got EOF from 127.7.39.2:51879 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 11 12:16:42 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:42 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:42.745281 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:42.736807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:50499)
0811 12:16:42.737406 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:42.737429 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:42.737580 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:42.737919 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:42.737924 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:42.738335 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:42.738548 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:42.738558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.739652 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.739656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:42.740616 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:42.740630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.741453 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.741459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:42.741671 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:42.742247 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:42.742266 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:42.744898 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":339,"threads_started":1}
W20260811 12:16:42.745412 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:42.736882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:50499 (local address 127.7.39.62:44485)
0811 12:16:42.737096 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:42.737103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:42.737532 (+ 429us) server_negotiation.cc:407] Connection header received
0811 12:16:42.737704 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:42.737708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:42.737774 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:42.737866 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:42.738747 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.739502 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.740769 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.741300 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.741445 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:42.745176 (+ 3731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:50499: BlockingRecv error: recv got EOF from 127.7.39.3:50499 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 12:16:42 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:42 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:42.801653 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:42.793211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:58477)
0811 12:16:42.793708 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:42.793724 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:42.793859 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:42.794191 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:42.794197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:42.794641 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:42.794809 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:42.794818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.795859 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.795864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:42.796815 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:42.796828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.797627 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.797633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:42.797798 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:42.798413 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:42.798438 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:42.801344 (+ 2906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":241,"threads_started":1}
W20260811 12:16:42.801911 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:42.793249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:58477 (local address 127.7.39.62:44485)
0811 12:16:42.793448 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:42.793454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:42.793811 (+ 357us) server_negotiation.cc:407] Connection header received
0811 12:16:42.793974 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:42.793980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:42.794046 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:42.794134 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:42.794960 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.795708 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.796962 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:42.797439 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:42.797593 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:42.801650 (+ 4057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:58477: BlockingRecv error: recv got EOF from 127.7.39.1:58477 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 12:16:43 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:43 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:43.295221 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:43.286072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:51891)
0811 12:16:43.286642 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:43.286660 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:43.286806 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:43.287131 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:43.287135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:43.287616 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:43.287808 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:43.287818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.288866 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.288870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:43.290085 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:43.290103 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.291077 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.291082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:43.291251 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:43.291970 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:43.291988 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:43.294975 (+ 2987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":253,"threads_started":1}
W20260811 12:16:43.295465 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:43.286358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:51891 (local address 127.7.39.62:44485)
0811 12:16:43.286574 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:43.286580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:43.286739 (+ 159us) server_negotiation.cc:407] Connection header received
0811 12:16:43.286925 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:43.286929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:43.286995 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:43.287101 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:43.287960 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.288720 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.290264 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.290878 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.291002 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:43.295217 (+ 4215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:51891: BlockingRecv error: recv got EOF from 127.7.39.2:51891 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 12:16:43 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:43 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:43.754935 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:43.746405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:54859)
0811 12:16:43.747057 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:43.747083 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:43.747259 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:43.747616 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:43.747621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:43.748079 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:43.748250 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:43.748260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.749367 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.749371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:43.750303 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:43.750314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.751063 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.751070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:43.751238 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:43.751803 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:43.751821 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:43.754600 (+ 2779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":354,"threads_started":1}
W20260811 12:16:43.755090 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:43.746551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:54859 (local address 127.7.39.62:44485)
0811 12:16:43.746766 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:43.746773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:43.747185 (+ 412us) server_negotiation.cc:407] Connection header received
0811 12:16:43.747386 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:43.747390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:43.747464 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:43.747557 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:43.748406 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.749222 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.750453 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.750874 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.750987 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:43.754863 (+ 3876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:54859: BlockingRecv error: recv got EOF from 127.7.39.3:54859 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 12:16:43 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:43 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:43.811918 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:43.802691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:59759)
0811 12:16:43.803185 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:43.803202 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:43.803340 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:43.803618 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:43.803621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:43.804000 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:43.804205 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:43.804215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.805377 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.805382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:43.806348 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:43.806362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.807127 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.807133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:43.807319 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:43.808019 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:43.808045 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:43.811573 (+ 3528us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":265,"threads_started":1}
W20260811 12:16:43.812067 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:43.802756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:59759 (local address 127.7.39.62:44485)
0811 12:16:43.803103 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:43.803109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:43.803277 (+ 168us) server_negotiation.cc:407] Connection header received
0811 12:16:43.803434 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:43.803438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:43.803502 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:43.803584 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:43.804354 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.805220 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.806472 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:43.806952 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:43.807078 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:43.811805 (+ 4727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:59759: BlockingRecv error: recv got EOF from 127.7.39.1:59759 (error 108)
Metrics: {"server-negotiator.queue_time_us":218}
Aug 11 12:16:44 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:44 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:44.305526 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:44.296307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:35583)
0811 12:16:44.296912 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:44.296929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:44.297110 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:44.297577 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:44.297582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:44.298014 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:44.298230 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:44.298241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.299323 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.299328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:44.300334 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:44.300350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.301461 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.301466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:44.301622 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:44.302336 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:44.302361 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:44.305260 (+ 2899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":349,"threads_started":1}
W20260811 12:16:44.305747 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:44.296500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:35583 (local address 127.7.39.62:44485)
0811 12:16:44.296762 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:44.296770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:44.297031 (+ 261us) server_negotiation.cc:407] Connection header received
0811 12:16:44.297295 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:44.297299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:44.297406 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:44.297516 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:44.298400 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.299162 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.300592 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.301303 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.301454 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:44.305517 (+ 4063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:35583: BlockingRecv error: recv got EOF from 127.7.39.2:35583 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 12:16:44 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:44 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.3@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:44.765827 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:44.756649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.3:52015)
0811 12:16:44.757274 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:44.757297 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:44.757481 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:44.757869 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:44.757874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:44.758305 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:44.758495 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:44.758505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.759556 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.759562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:44.760765 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:44.760782 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.761639 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.761646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:44.761828 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:44.762484 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:44.762503 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:44.765526 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":316,"threads_started":1}
W20260811 12:16:44.766181 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:44.756801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:52015 (local address 127.7.39.62:44485)
0811 12:16:44.757037 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:44.757043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:44.757383 (+ 340us) server_negotiation.cc:407] Connection header received
0811 12:16:44.757614 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:44.757620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:44.757698 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:44.757798 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:44.758645 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.759395 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.760911 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.761456 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.761596 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:44.765890 (+ 4294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.3:52015: BlockingRecv error: recv got EOF from 127.7.39.3:52015 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 12:16:44 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:44 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.1@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:44.826853 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:44.813234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.1:57773)
0811 12:16:44.813752 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:44.813772 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:44.813895 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:44.814203 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:44.814206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:44.814638 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:44.814845 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:44.814854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.815980 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.815985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:44.816967 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:44.816982 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.817849 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.817854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:44.817986 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:44.823049 (+ 5063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:44.823079 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:44.826452 (+ 3373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":287,"threads_started":1}
W20260811 12:16:44.826942 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:44.813292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:57773 (local address 127.7.39.62:44485)
0811 12:16:44.813467 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:44.813474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:44.813880 (+ 406us) server_negotiation.cc:407] Connection header received
0811 12:16:44.814010 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:44.814014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:44.814083 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:44.814171 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:44.815006 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.815786 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.817175 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:44.817687 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:44.817801 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:44.826696 (+ 8895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.1:57773: BlockingRecv error: recv got EOF from 127.7.39.1:57773 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:16:45 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:16:45 dist-test-slave-rmhg krb5kdc[20478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.39.2@KRBTEST.COM for oryx/127.7.39.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:16:45.320775 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:45.306914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44485 (local address 127.7.39.2:47301)
0811 12:16:45.307587 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:45.307606 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:45.307769 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:45.308150 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:45.308154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:45.308723 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:45.308970 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:45.308982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.310103 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.310108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:45.311051 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:45.311065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.311933 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.311938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:45.312077 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:45.317240 (+ 5163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:45.317265 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:45.320423 (+ 3158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.39.62:44485: Server oryx/127.7.39.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":420,"threads_started":1}
W20260811 12:16:45.321017 21979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:45.307049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:47301 (local address 127.7.39.62:44485)
0811 12:16:45.307238 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:45.307245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:45.307704 (+ 459us) server_negotiation.cc:407] Connection header received
0811 12:16:45.307894 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:45.307899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:45.307991 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:45.308106 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:45.309190 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.309952 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.311287 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.311769 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.311883 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:45.320734 (+ 8851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.39.2:47301: BlockingRecv error: recv got EOF from 127.7.39.2:47301 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:16:45.605896 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 21365
I20260811 12:16:45.618397 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:45.607212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35458 (local address 127.7.39.1:46651)
0811 12:16:45.607794 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:45.607802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:45.607831 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:16:45.607893 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:45.607899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:45.608151 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:45.608311 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:45.609843 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.610894 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.612111 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.612673 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.612771 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:45.615262 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:45.615272 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:45.615287 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:45.615327 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:45.617885 (+ 2558us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.39.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:16:45.618153 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35458: Request ticket server kudu/127.7.39.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":300,"threads_started":1}
W20260811 12:16:45.618503 22057 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35458: Request ticket server kudu/127.7.39.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:16:45.618597 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 21365: Not authorized: Client connection negotiation failed: client connection to 127.7.39.1:46651: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.39.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:16:45.618841 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 21365
I20260811 12:16:45.652699 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 21581
I20260811 12:16:45.664975 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:45.653747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54786 (local address 127.7.39.2:40183)
0811 12:16:45.654347 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:45.654355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:45.654387 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:16:45.654459 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:45.654466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:45.654698 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:45.654855 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:45.656149 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.657459 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.658749 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.659345 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.659443 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:45.661930 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:45.661939 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:45.661957 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:45.661994 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:45.664419 (+ 2425us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.39.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:16:45.664727 (+ 308us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54786: Request ticket server kudu/127.7.39.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":263,"threads_started":1}
W20260811 12:16:45.665095 22058 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54786: Request ticket server kudu/127.7.39.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:16:45.665223 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 21581: Not authorized: Client connection negotiation failed: client connection to 127.7.39.2:40183: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.39.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:16:45.665385 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 21581
I20260811 12:16:45.686342 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 21781
I20260811 12:16:45.698972 22062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:45.687409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52096 (local address 127.7.39.3:41513)
0811 12:16:45.688077 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:45.688085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:45.688113 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:16:45.688178 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:45.688186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:45.688456 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:45.688627 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:45.690192 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.691345 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.692657 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:45.693272 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:45.693393 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:45.695852 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:45.695862 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:45.695877 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:45.695924 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:45.698303 (+ 2379us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.39.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:16:45.698650 (+ 347us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52096: Request ticket server kudu/127.7.39.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":376,"threads_started":1}
W20260811 12:16:45.699080 22062 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52096: Request ticket server kudu/127.7.39.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:16:45.699069 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 21781: Not authorized: Client connection negotiation failed: client connection to 127.7.39.3:41513: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.39.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:16:45.699263 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 21781
I20260811 12:16:45.736546 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 21251
I20260811 12:16:45.738029 21352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:45.860014 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 21251
2026-08-11T12:16:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21650 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T12:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:45Z Disabled control of system clock
I20260811 12:16:45.953848 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40273
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:35675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:40273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 12:16:46.301118 22070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:46.301477 22070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:46.301604 22070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:46.309123 22070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:46.309263 22070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:46.309326 22070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:46.309405 22070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:46.309476 22070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:46.319363 22070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:40273
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:46.320798 22070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:46.322844 22070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:46.335377 22083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:46.335457 22087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:46.335431 22082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:46.336057 22070 server_base.cc:1034] running on GCE node
I20260811 12:16:46.336899 22070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:46.338423 22070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:46.339648 22070 hybrid_clock.cc:648] HybridClock initialized: now 1786450606339592 us; error 48 us; skew 500 ppm
I20260811 12:16:46.340020 22070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:16:46.342442 22070 webserver.cc:460] Webserver started at http://127.7.39.62:43753/ using document root <none> and password file <none>
I20260811 12:16:46.343196 22070 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:46.343303 22070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:46.343674 22070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:46.346968 22070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "373e353d8b044c28ba5a27d61036aa7f"
format_stamp: "Formatted at 2026-08-11 12:16:46 on dist-test-slave-rmhg"
server_key: "116c2f8ece85f31d54be1a2702cd200a"
server_key_iv: "fc34e76bfe2272a892b92ed21d4f3fc6"
server_key_version: "encryptionkey@0"
I20260811 12:16:46.347842 22070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "373e353d8b044c28ba5a27d61036aa7f"
format_stamp: "Formatted at 2026-08-11 12:16:46 on dist-test-slave-rmhg"
server_key: "116c2f8ece85f31d54be1a2702cd200a"
server_key_iv: "fc34e76bfe2272a892b92ed21d4f3fc6"
server_key_version: "encryptionkey@0"
I20260811 12:16:46.353669 22070 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:16:46.358003 22095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:46.359628 22070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:16:46.359833 22070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "373e353d8b044c28ba5a27d61036aa7f"
format_stamp: "Formatted at 2026-08-11 12:16:46 on dist-test-slave-rmhg"
server_key: "116c2f8ece85f31d54be1a2702cd200a"
server_key_iv: "fc34e76bfe2272a892b92ed21d4f3fc6"
server_key_version: "encryptionkey@0"
I20260811 12:16:46.360077 22070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:46.374704 22070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:46.375818 22070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:46.376160 22070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:46.396010 22070 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:40273
I20260811 12:16:46.396040 22176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:40273 every 8 connection(s)
I20260811 12:16:46.397533 22070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:16:46.400584 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 22070
I20260811 12:16:46.400791 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:16:46.401500 7324 external_mini_cluster.cc:1428] Setting key 3b4605a4e4afd9377e94300d28e70a20
I20260811 12:16:46.401983 22177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:46.418339 22177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f: Bootstrap starting.
I20260811 12:16:46.418856 22183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:46.403688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47432 (local address 127.7.39.62:40273)
0811 12:16:46.404891 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:46.404924 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:46.405022 (+ 98us) server_negotiation.cc:407] Connection header received
0811 12:16:46.407237 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:46.407276 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:46.408027 (+ 751us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:46.408576 (+ 549us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:46.409237 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:46.411318 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:46.412878 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:46.414001 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:46.414216 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:46.415497 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:46.415511 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:46.415540 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:46.416381 (+ 841us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:46.416514 (+ 133us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:46.416545 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:46.416690 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:46.416978 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 12:16:46.417404 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":361,"threads_started":1}
I20260811 12:16:46.422036 22177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:46.423132 22177 log.cc:824] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:46.426463 22177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f: No bootstrap required, opened a new log
I20260811 12:16:46.434015 22177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373e353d8b044c28ba5a27d61036aa7f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40273 } }
I20260811 12:16:46.434437 22177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:46.434547 22177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373e353d8b044c28ba5a27d61036aa7f, State: Initialized, Role: FOLLOWER
I20260811 12:16:46.435158 22177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373e353d8b044c28ba5a27d61036aa7f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40273 } }
I20260811 12:16:46.435391 22177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:16:46.435530 22177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:16:46.435698 22177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:46.438095 22177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373e353d8b044c28ba5a27d61036aa7f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40273 } }
I20260811 12:16:46.438651 22177 leader_election.cc:302] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 373e353d8b044c28ba5a27d61036aa7f; no voters:
I20260811 12:16:46.439307 22177 leader_election.cc:288] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:16:46.439452 22186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:46.440186 22186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [term 1 LEADER]: Becoming Leader. State: Replica: 373e353d8b044c28ba5a27d61036aa7f, State: Running, Role: LEADER
I20260811 12:16:46.440743 22186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373e353d8b044c28ba5a27d61036aa7f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40273 } }
I20260811 12:16:46.441612 22177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:46.443727 22189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "373e353d8b044c28ba5a27d61036aa7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373e353d8b044c28ba5a27d61036aa7f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40273 } } }
I20260811 12:16:46.444155 22189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:46.443567 22191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 373e353d8b044c28ba5a27d61036aa7f. Latest consensus state: current_term: 1 leader_uuid: "373e353d8b044c28ba5a27d61036aa7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373e353d8b044c28ba5a27d61036aa7f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40273 } } }
I20260811 12:16:46.444397 22191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:46.446848 22198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:46.452323 22198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:46.462196 22198 catalog_manager.cc:1347] Generated new cluster ID: 9b2163d96e824938920b0fa7f2c60360
I20260811 12:16:46.462330 22198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:46.472338 22198 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:16:46.473737 22198 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:46.479787 22198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 373e353d8b044c28ba5a27d61036aa7f: Generated new TSK 0
I20260811 12:16:46.480645 22198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:16:46.491652 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40273
--builtin_ntp_servers=127.7.39.20:35675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 12:16:46.846179 22215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:46.846481 22215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:46.846616 22215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:46.854208 22215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:46.854326 22215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:46.854467 22215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:46.863937 22215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:46.865406 22215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:46.867230 22215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:46.880182 22223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:46.880661 22225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:46.881336 22215 server_base.cc:1034] running on GCE node
W20260811 12:16:46.881434 22222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:46.882129 22215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:46.883114 22215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:46.884341 22215 hybrid_clock.cc:648] HybridClock initialized: now 1786450606884280 us; error 50 us; skew 500 ppm
I20260811 12:16:46.884688 22215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:16:46.887109 22215 webserver.cc:460] Webserver started at http://127.7.39.1:33287/ using document root <none> and password file <none>
I20260811 12:16:46.887845 22215 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:46.887950 22215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:46.888269 22215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:46.891225 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "9361fcf7cfbf4fb4859408dedcba2704"
format_stamp: "Formatted at 2026-08-11 12:16:46 on dist-test-slave-rmhg"
server_key: "58e5079fc44ee3a691db279fa538ee92"
server_key_iv: "69f7f8b8ff23a5d8a095bd649b99a704"
server_key_version: "encryptionkey@0"
I20260811 12:16:46.892030 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "9361fcf7cfbf4fb4859408dedcba2704"
format_stamp: "Formatted at 2026-08-11 12:16:46 on dist-test-slave-rmhg"
server_key: "58e5079fc44ee3a691db279fa538ee92"
server_key_iv: "69f7f8b8ff23a5d8a095bd649b99a704"
server_key_version: "encryptionkey@0"
I20260811 12:16:46.897514 22215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 12:16:46.901377 22235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:46.902942 22215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:46.903144 22215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "9361fcf7cfbf4fb4859408dedcba2704"
format_stamp: "Formatted at 2026-08-11 12:16:46 on dist-test-slave-rmhg"
server_key: "58e5079fc44ee3a691db279fa538ee92"
server_key_iv: "69f7f8b8ff23a5d8a095bd649b99a704"
server_key_version: "encryptionkey@0"
I20260811 12:16:46.903381 22215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:46.930696 22215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:46.931687 22215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:46.931963 22215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:46.932979 22215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:46.934571 22215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:46.934674 22215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:46.934790 22215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:46.934866 22215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:46.966889 22215 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:42521
I20260811 12:16:46.966951 22399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:42521 every 8 connection(s)
I20260811 12:16:46.968214 22215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:46.971791 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 22215
I20260811 12:16:46.972095 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:16:46.973081 7324 external_mini_cluster.cc:1428] Setting key 72cf2db5ee64c98cbbf10db58f12c4b8
I20260811 12:16:46.976425 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40273
--builtin_ntp_servers=127.7.39.20:35675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 12:16:46.982797 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:46.970999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:47519 (local address 127.7.39.62:40273)
0811 12:16:46.971480 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:46.971485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:46.973232 (+ 1747us) server_negotiation.cc:407] Connection header received
0811 12:16:46.975406 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:46.975410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:46.975484 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:46.975595 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:46.976757 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:46.977874 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:46.979709 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:46.980260 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:46.980384 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:46.981895 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:46.981902 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:46.981906 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:46.981930 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:46.982010 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:46.982029 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:46.982115 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:46.982342 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 12:16:46.982452 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":255,"threads_started":1}
I20260811 12:16:46.983785 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:46.971897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40273 (local address 127.7.39.1:47519)
0811 12:16:46.973039 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:46.973082 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:46.975106 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:46.975829 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:46.975838 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:46.975937 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:46.976567 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:46.976596 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:46.978047 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:46.978052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:46.979557 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:46.979570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:46.980499 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:46.980505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:46.980766 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:46.981529 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:16:46.981532 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:46.981593 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:16:46.981687 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:46.982132 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:46.982141 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:46.982148 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:16:46.982317 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 12:16:46.982658 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":345,"threads_started":1}
I20260811 12:16:46.985431 22400 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40273
I20260811 12:16:46.985749 22400 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:46.986526 22400 heartbeater.cc:511] Master 127.7.39.62:40273 requested a full tablet report, sending...
I20260811 12:16:46.988637 22123 ts_manager.cc:194] Registered new tserver with Master: 9361fcf7cfbf4fb4859408dedcba2704 (127.7.39.1:42521)
I20260811 12:16:46.990201 22123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.1:47519
W20260811 12:16:47.333251 22406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:47.333591 22406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:47.333721 22406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:47.341300 22406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:47.341424 22406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:47.341580 22406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:47.351245 22406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:47.352649 22406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:47.354661 22406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:47.368945 22420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:47.369005 22415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:47.369007 22416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:47.369464 22406 server_base.cc:1034] running on GCE node
I20260811 12:16:47.370357 22406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:47.371400 22406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:47.372654 22406 hybrid_clock.cc:648] HybridClock initialized: now 1786450607372581 us; error 63 us; skew 500 ppm
I20260811 12:16:47.373013 22406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:16:47.375408 22406 webserver.cc:460] Webserver started at http://127.7.39.2:32859/ using document root <none> and password file <none>
I20260811 12:16:47.376164 22406 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:47.376287 22406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:47.376616 22406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:47.379691 22406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "21057df1288b45e69eb1791ac4edadc1"
format_stamp: "Formatted at 2026-08-11 12:16:47 on dist-test-slave-rmhg"
server_key: "c0f397c9e83b2832cdb1665b98a91e32"
server_key_iv: "19a0e4ab81cc833c37c681f974eddd1b"
server_key_version: "encryptionkey@0"
I20260811 12:16:47.380529 22406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "21057df1288b45e69eb1791ac4edadc1"
format_stamp: "Formatted at 2026-08-11 12:16:47 on dist-test-slave-rmhg"
server_key: "c0f397c9e83b2832cdb1665b98a91e32"
server_key_iv: "19a0e4ab81cc833c37c681f974eddd1b"
server_key_version: "encryptionkey@0"
I20260811 12:16:47.386109 22406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:16:47.389910 22427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:47.391441 22406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:47.391644 22406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "21057df1288b45e69eb1791ac4edadc1"
format_stamp: "Formatted at 2026-08-11 12:16:47 on dist-test-slave-rmhg"
server_key: "c0f397c9e83b2832cdb1665b98a91e32"
server_key_iv: "19a0e4ab81cc833c37c681f974eddd1b"
server_key_version: "encryptionkey@0"
I20260811 12:16:47.391878 22406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:47.415414 22406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:47.416371 22406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:47.416668 22406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:47.417814 22406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:47.419404 22406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:47.419513 22406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:47.419625 22406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:47.419703 22406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:47.451493 22406 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:33097
I20260811 12:16:47.451515 22616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:33097 every 8 connection(s)
I20260811 12:16:47.452864 22406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:47.458274 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 22406
I20260811 12:16:47.458446 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:16:47.458995 7324 external_mini_cluster.cc:1428] Setting key ead9bde3c2110218e79b4c71b2833418
I20260811 12:16:47.462033 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40273
--builtin_ntp_servers=127.7.39.20:35675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 12:16:47.467299 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:47.455643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:48595 (local address 127.7.39.62:40273)
0811 12:16:47.455848 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:47.455853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:47.457285 (+ 1432us) server_negotiation.cc:407] Connection header received
0811 12:16:47.459617 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:47.459623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:47.459716 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:47.459851 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:47.461021 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.461991 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.464321 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.464814 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.464954 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:47.466555 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:47.466560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:47.466563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:47.466582 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:47.466616 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:47.466629 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:47.466701 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:47.466973 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 12:16:47.467060 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:16:47.468670 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:47.456128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40273 (local address 127.7.39.2:48595)
0811 12:16:47.457084 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:47.457133 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:47.459241 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:47.460109 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:47.460121 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:47.460224 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:47.460849 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:47.460875 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.462176 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.462181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:47.464181 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:47.464197 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.465055 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.465062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:47.465392 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:47.466149 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:16:47.466152 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:47.466204 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:16:47.466314 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:47.466721 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:47.466731 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:47.466740 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:16:47.466924 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 12:16:47.467349 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":295,"threads_started":1}
I20260811 12:16:47.470245 22617 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40273
I20260811 12:16:47.470584 22617 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:47.471516 22617 heartbeater.cc:511] Master 127.7.39.62:40273 requested a full tablet report, sending...
I20260811 12:16:47.473212 22123 ts_manager.cc:194] Registered new tserver with Master: 21057df1288b45e69eb1791ac4edadc1 (127.7.39.2:33097)
I20260811 12:16:47.474223 22123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.2:48595
W20260811 12:16:47.812518 22620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:47.812842 22620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:47.812937 22620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:47.820739 22620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:47.820884 22620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:47.821065 22620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:47.831102 22620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:47.832466 22620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:47.834316 22620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:47.847829 22627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:47.847882 22628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:47.847791 22631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:47.848547 22620 server_base.cc:1034] running on GCE node
I20260811 12:16:47.849318 22620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:47.850414 22620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:47.851621 22620 hybrid_clock.cc:648] HybridClock initialized: now 1786450607851553 us; error 66 us; skew 500 ppm
I20260811 12:16:47.851940 22620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:16:47.854281 22620 webserver.cc:460] Webserver started at http://127.7.39.3:41133/ using document root <none> and password file <none>
I20260811 12:16:47.854995 22620 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:47.855106 22620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:47.855446 22620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:47.858507 22620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "1dc0cc2c316b4f9ea62c86ae664329fd"
format_stamp: "Formatted at 2026-08-11 12:16:47 on dist-test-slave-rmhg"
server_key: "8f8b13406b6928a80e1c9ed0b7e50f51"
server_key_iv: "273ff59427d8ece3584fd1dbdb258b83"
server_key_version: "encryptionkey@0"
I20260811 12:16:47.859308 22620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "1dc0cc2c316b4f9ea62c86ae664329fd"
format_stamp: "Formatted at 2026-08-11 12:16:47 on dist-test-slave-rmhg"
server_key: "8f8b13406b6928a80e1c9ed0b7e50f51"
server_key_iv: "273ff59427d8ece3584fd1dbdb258b83"
server_key_version: "encryptionkey@0"
I20260811 12:16:47.864928 22620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:16:47.868592 22645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:47.870000 22620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:47.870172 22620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "1dc0cc2c316b4f9ea62c86ae664329fd"
format_stamp: "Formatted at 2026-08-11 12:16:47 on dist-test-slave-rmhg"
server_key: "8f8b13406b6928a80e1c9ed0b7e50f51"
server_key_iv: "273ff59427d8ece3584fd1dbdb258b83"
server_key_version: "encryptionkey@0"
I20260811 12:16:47.870379 22620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:47.886006 22620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:47.886906 22620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:47.887195 22620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:47.888198 22620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:47.889914 22620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:47.890017 22620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:47.890129 22620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:47.890208 22620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:47.922497 22620 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:40665
I20260811 12:16:47.922528 22825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:40665 every 8 connection(s)
I20260811 12:16:47.923929 22620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:47.933130 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 22620
I20260811 12:16:47.933346 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:16:47.933801 7324 external_mini_cluster.cc:1428] Setting key a5a1396a414302822436b4fa9dcf257b
I20260811 12:16:47.937458 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:47.926676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:33447 (local address 127.7.39.62:40273)
0811 12:16:47.926852 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:47.926856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:47.928413 (+ 1557us) server_negotiation.cc:407] Connection header received
0811 12:16:47.930409 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:47.930414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:47.930493 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:47.930616 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:47.931863 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.932647 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.934468 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.935046 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.935180 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:47.936658 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:47.936664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:47.936668 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:47.936690 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:47.936731 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:47.936748 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:47.936844 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:47.937076 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 12:16:47.937200 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:16:47.938651 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:47.927211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40273 (local address 127.7.39.3:33447)
0811 12:16:47.928240 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:47.928280 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:47.930084 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:47.930858 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:47.930867 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:47.930973 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:47.931655 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:47.931687 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.932830 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.932834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:47.934297 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:47.934315 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.935310 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.935316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:47.935536 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:47.936319 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:16:47.936321 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:47.936367 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:16:47.936455 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:47.936854 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:47.936863 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:47.936870 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:16:47.937057 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 12:16:47.937472 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":315,"threads_started":1}
I20260811 12:16:47.940147 22830 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40273
I20260811 12:16:47.940439 22830 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:47.941310 22830 heartbeater.cc:511] Master 127.7.39.62:40273 requested a full tablet report, sending...
I20260811 12:16:47.942881 22123 ts_manager.cc:194] Registered new tserver with Master: 1dc0cc2c316b4f9ea62c86ae664329fd (127.7.39.3:40665)
I20260811 12:16:47.943696 22123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.3:33447
I20260811 12:16:47.949357 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:16:47.955693 22403 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:47.953952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47440 (local address 127.7.39.62:40273)
0811 12:16:47.954153 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:47.954158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:47.954612 (+ 454us) server_negotiation.cc:407] Connection header received
0811 12:16:47.954778 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:47.954781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:47.954855 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:47.954941 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:47.955408 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47440: BlockingRecv error: recv got EOF from 127.0.0.1:47440 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:16:47.956652 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 22215
I20260811 12:16:47.964666 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:47.957621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.7.39.1:42521)
0811 12:16:47.958168 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:47.958175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:47.958200 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:16:47.958256 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:47.958262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:47.958471 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:47.958613 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:47.959025 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.960023 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.961337 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:47.961901 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:47.962021 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:47.963035 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:47.963051 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:47.963066 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:47.963713 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:47.963807 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:47.963831 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:47.963939 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:47.964127 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 12:16:47.964303 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":237,"threads_started":1}
I20260811 12:16:47.966013 22393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:48.048975 22400 heartbeater.cc:503] Master 127.7.39.62:40273 was elected leader, sending a full tablet report...
I20260811 12:16:48.050379 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 22215
W20260811 12:16:48.076414 22104 connection.cc:535] server connection from 127.7.39.1:47519 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 12:16:48.076627 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 22406
I20260811 12:16:48.084798 22846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:48.077615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.7.39.2:33097)
0811 12:16:48.078182 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:48.078190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:48.078219 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:16:48.078282 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:48.078289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:48.078509 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:48.078663 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:48.079102 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:48.080147 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:48.081458 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:48.082051 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:48.082173 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:48.083298 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:48.083307 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:48.083321 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:48.083948 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:48.083999 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:48.084029 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:48.084132 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:48.084307 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 12:16:48.084463 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":292,"threads_started":1}
I20260811 12:16:48.086143 22612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:48.169260 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 22406
I20260811 12:16:48.192304 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 22620
I20260811 12:16:48.201653 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:48.193487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37496 (local address 127.7.39.3:40665)
0811 12:16:48.194190 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:48.194198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:48.194231 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:16:48.194299 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:48.194306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:48.194541 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:48.194716 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:48.195201 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:48.196311 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:48.197933 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:48.198580 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:48.198709 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:48.199928 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:48.199938 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:48.199954 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:48.200612 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:48.200672 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:48.200694 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:48.200800 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:48.200974 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 12:16:48.201186 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":361,"threads_started":1}
I20260811 12:16:48.203176 22813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:48.285333 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 22620
I20260811 12:16:48.311705 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 22070
I20260811 12:16:48.313024 22172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:48.420305 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 22070
2026-08-11T12:16:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2557 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T12:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:48Z Disabled control of system clock
I20260811 12:16:48.512329 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:42705
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:42705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 12:16:48.872102 22859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:48.872849 22859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:48.873237 22859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:48.881099 22859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:48.881280 22859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:48.881359 22859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:48.881448 22859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:48.881503 22859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:48.891346 22859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:42705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:42705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:48.892827 22859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:48.894824 22859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:48.907069 22870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:48.907110 22876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:48.907167 22871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:48.908069 22859 server_base.cc:1034] running on GCE node
I20260811 12:16:48.908932 22859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:48.910532 22859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:48.911762 22859 hybrid_clock.cc:648] HybridClock initialized: now 1786450608911721 us; error 39 us; skew 500 ppm
I20260811 12:16:48.912133 22859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:16:48.914656 22859 webserver.cc:460] Webserver started at http://127.7.39.62:43119/ using document root <none> and password file <none>
I20260811 12:16:48.915449 22859 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:48.915555 22859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:48.915894 22859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:48.919003 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "ae9a67b70af5499aacd7d229ff78a336"
format_stamp: "Formatted at 2026-08-11 12:16:48 on dist-test-slave-rmhg"
server_key: "da84392d1c45fc3e0adb49fcf5e3c661"
server_key_iv: "be62ed0f8249cb4cf2098a2c5f0e62e6"
server_key_version: "encryptionkey@0"
I20260811 12:16:48.919863 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "ae9a67b70af5499aacd7d229ff78a336"
format_stamp: "Formatted at 2026-08-11 12:16:48 on dist-test-slave-rmhg"
server_key: "da84392d1c45fc3e0adb49fcf5e3c661"
server_key_iv: "be62ed0f8249cb4cf2098a2c5f0e62e6"
server_key_version: "encryptionkey@0"
I20260811 12:16:48.925534 22859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 12:16:48.929252 22895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:48.930737 22859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:48.930912 22859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "ae9a67b70af5499aacd7d229ff78a336"
format_stamp: "Formatted at 2026-08-11 12:16:48 on dist-test-slave-rmhg"
server_key: "da84392d1c45fc3e0adb49fcf5e3c661"
server_key_iv: "be62ed0f8249cb4cf2098a2c5f0e62e6"
server_key_version: "encryptionkey@0"
I20260811 12:16:48.931129 22859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:48.937466 22859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:48.938301 22859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:48.938594 22859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:48.956353 22859 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:42705
I20260811 12:16:48.956372 22978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:42705 every 8 connection(s)
I20260811 12:16:48.957826 22859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:16:48.961984 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 22859
I20260811 12:16:48.962240 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:16:48.962546 22980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:48.963111 7324 external_mini_cluster.cc:1428] Setting key f0ae1307366fd61420f163d6dfc9ec4b
I20260811 12:16:48.973690 22984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:48.965581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.7.39.62:42705)
0811 12:16:48.966873 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:48.966886 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:48.966943 (+ 57us) server_negotiation.cc:407] Connection header received
0811 12:16:48.969570 (+ 2627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:48.969603 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:48.970138 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:48.970590 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:48.971772 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:48.971780 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:48.971796 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:48.971813 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:48.971883 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:48.971901 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:48.972133 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:48.972279 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:16:48.972569 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":468,"threads_started":1}
I20260811 12:16:48.977028 22980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336: Bootstrap starting.
I20260811 12:16:48.980209 22980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:48.981344 22980 log.cc:824] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:48.984468 22980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336: No bootstrap required, opened a new log
I20260811 12:16:48.992291 22980 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9a67b70af5499aacd7d229ff78a336" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42705 } }
I20260811 12:16:48.992715 22980 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:48.992820 22980 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9a67b70af5499aacd7d229ff78a336, State: Initialized, Role: FOLLOWER
I20260811 12:16:48.993496 22980 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9a67b70af5499aacd7d229ff78a336" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42705 } }
I20260811 12:16:48.993733 22980 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:16:48.993866 22980 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:16:48.994036 22980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:48.996382 22980 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9a67b70af5499aacd7d229ff78a336" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42705 } }
I20260811 12:16:48.996928 22980 leader_election.cc:302] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae9a67b70af5499aacd7d229ff78a336; no voters:
I20260811 12:16:48.997613 22980 leader_election.cc:288] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:16:48.997757 22986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:48.998461 22986 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [term 1 LEADER]: Becoming Leader. State: Replica: ae9a67b70af5499aacd7d229ff78a336, State: Running, Role: LEADER
I20260811 12:16:48.999068 22986 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9a67b70af5499aacd7d229ff78a336" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42705 } }
I20260811 12:16:48.999759 22980 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:49.002270 22989 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae9a67b70af5499aacd7d229ff78a336. Latest consensus state: current_term: 1 leader_uuid: "ae9a67b70af5499aacd7d229ff78a336" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9a67b70af5499aacd7d229ff78a336" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42705 } } }
I20260811 12:16:49.002738 22989 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:49.003950 22988 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae9a67b70af5499aacd7d229ff78a336" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9a67b70af5499aacd7d229ff78a336" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42705 } } }
I20260811 12:16:49.004393 22988 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:49.010684 22997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:49.016105 22997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:49.026602 22997 catalog_manager.cc:1347] Generated new cluster ID: 1ca1239e8f154a1a9e7e0b8085ff1602
I20260811 12:16:49.026744 22997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:49.047538 22997 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:16:49.048010 22997 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:49.057925 22997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae9a67b70af5499aacd7d229ff78a336: Generated new TSK 0
I20260811 12:16:49.059135 22997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:16:49.072808 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42705
--builtin_ntp_servers=127.7.39.20:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 12:16:49.432528 23014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:49.432835 23014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:49.432938 23014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:49.440586 23014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:49.440721 23014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:49.440845 23014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:49.450709 23014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:49.452219 23014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:49.454241 23014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:49.468127 23021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:49.468168 23025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:49.468379 23022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:49.468756 23014 server_base.cc:1034] running on GCE node
I20260811 12:16:49.469614 23014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:49.470683 23014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:49.471899 23014 hybrid_clock.cc:648] HybridClock initialized: now 1786450609471845 us; error 44 us; skew 500 ppm
I20260811 12:16:49.472270 23014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:16:49.474702 23014 webserver.cc:460] Webserver started at http://127.7.39.1:34381/ using document root <none> and password file <none>
I20260811 12:16:49.475430 23014 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:49.475528 23014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:49.475864 23014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:49.481467 23014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "acad9dbfc0264fb0aeb184a9edf3b839"
format_stamp: "Formatted at 2026-08-11 12:16:49 on dist-test-slave-rmhg"
server_key: "d96f90e396de6066b5f3db4b53753962"
server_key_iv: "c55f5a521e89eccf342441117c2ef0a5"
server_key_version: "encryptionkey@0"
I20260811 12:16:49.482342 23014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "acad9dbfc0264fb0aeb184a9edf3b839"
format_stamp: "Formatted at 2026-08-11 12:16:49 on dist-test-slave-rmhg"
server_key: "d96f90e396de6066b5f3db4b53753962"
server_key_iv: "c55f5a521e89eccf342441117c2ef0a5"
server_key_version: "encryptionkey@0"
I20260811 12:16:49.488099 23014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:16:49.492039 23034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:49.493701 23014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:49.493885 23014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "acad9dbfc0264fb0aeb184a9edf3b839"
format_stamp: "Formatted at 2026-08-11 12:16:49 on dist-test-slave-rmhg"
server_key: "d96f90e396de6066b5f3db4b53753962"
server_key_iv: "c55f5a521e89eccf342441117c2ef0a5"
server_key_version: "encryptionkey@0"
I20260811 12:16:49.494140 23014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:49.500396 23014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:49.501420 23014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:49.501699 23014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:49.502766 23014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:49.504468 23014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:49.504562 23014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:49.504673 23014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:49.504747 23014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:49.536639 23014 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:42197
I20260811 12:16:49.536684 23213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:42197 every 8 connection(s)
I20260811 12:16:49.538092 23014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:49.543793 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 23014
I20260811 12:16:49.544008 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:16:49.544744 7324 external_mini_cluster.cc:1428] Setting key f345bac9bcf44a4c9fd9f161795f1348
I20260811 12:16:49.547297 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:49.540845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:53453 (local address 127.7.39.62:42705)
0811 12:16:49.541384 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:49.541391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:49.542693 (+ 1302us) server_negotiation.cc:407] Connection header received
0811 12:16:49.544543 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:49.544550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:49.544640 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:49.544782 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:49.546418 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:49.546423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:49.546426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:49.546445 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:49.546532 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:49.546546 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:49.546680 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:49.546913 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 12:16:49.546975 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":247,"threads_started":1}
I20260811 12:16:49.548026 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42705
--builtin_ntp_servers=127.7.39.20:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 12:16:49.548362 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:49.541511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42705 (local address 127.7.39.1:53453)
0811 12:16:49.542511 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:49.542554 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:49.544220 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:49.545057 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:49.545067 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:49.545219 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:49.545972 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:16:49.545976 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:49.546034 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:16:49.546171 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:49.546660 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:49.546677 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:49.546684 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:16:49.547000 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 12:16:49.547269 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":285,"threads_started":1}
I20260811 12:16:49.550019 23214 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42705
I20260811 12:16:49.550334 23214 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:49.550971 23214 heartbeater.cc:511] Master 127.7.39.62:42705 requested a full tablet report, sending...
I20260811 12:16:49.553118 22925 ts_manager.cc:194] Registered new tserver with Master: acad9dbfc0264fb0aeb184a9edf3b839 (127.7.39.1:42197)
W20260811 12:16:49.905810 23219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:49.906137 23219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:49.906275 23219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:49.913797 23219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:49.913931 23219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:49.914081 23219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:49.923702 23219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:49.925191 23219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:49.927173 23219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:49.941510 23233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:49.941409 23234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:49.941437 23236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:49.942418 23219 server_base.cc:1034] running on GCE node
I20260811 12:16:49.943157 23219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:49.944172 23219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:49.945421 23219 hybrid_clock.cc:648] HybridClock initialized: now 1786450609945348 us; error 57 us; skew 500 ppm
I20260811 12:16:49.945770 23219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:16:49.948180 23219 webserver.cc:460] Webserver started at http://127.7.39.2:35949/ using document root <none> and password file <none>
I20260811 12:16:49.948905 23219 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:49.949016 23219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:49.949395 23219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:49.952401 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "6890954f724d411a97f37d9f6df740c5"
format_stamp: "Formatted at 2026-08-11 12:16:49 on dist-test-slave-rmhg"
server_key: "8d6853a37d9a65081ef2b884174a4d29"
server_key_iv: "f06315deb490fe23880d812c8fcf6390"
server_key_version: "encryptionkey@0"
I20260811 12:16:49.953317 23219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "6890954f724d411a97f37d9f6df740c5"
format_stamp: "Formatted at 2026-08-11 12:16:49 on dist-test-slave-rmhg"
server_key: "8d6853a37d9a65081ef2b884174a4d29"
server_key_iv: "f06315deb490fe23880d812c8fcf6390"
server_key_version: "encryptionkey@0"
I20260811 12:16:49.959806 23219 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 12:16:49.964602 23244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:49.966454 23219 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:16:49.966637 23219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "6890954f724d411a97f37d9f6df740c5"
format_stamp: "Formatted at 2026-08-11 12:16:49 on dist-test-slave-rmhg"
server_key: "8d6853a37d9a65081ef2b884174a4d29"
server_key_iv: "f06315deb490fe23880d812c8fcf6390"
server_key_version: "encryptionkey@0"
I20260811 12:16:49.966892 23219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:49.974759 23219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:49.975860 23219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:49.976181 23219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:49.977427 23219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:49.978941 23219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:49.979029 23219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:49.979125 23219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:49.979179 23219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:50.011157 23219 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:33545
I20260811 12:16:50.011212 23415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:33545 every 8 connection(s)
I20260811 12:16:50.012593 23219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:50.019469 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 23219
I20260811 12:16:50.019642 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:16:50.020079 7324 external_mini_cluster.cc:1428] Setting key a742798957b04f2234d892ae3d606703
I20260811 12:16:50.021136 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:50.015264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:37603 (local address 127.7.39.62:42705)
0811 12:16:50.015447 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:50.015452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:50.016919 (+ 1467us) server_negotiation.cc:407] Connection header received
0811 12:16:50.018786 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:50.018792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:50.018863 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:50.018957 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:50.020383 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:50.020388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:50.020390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:50.020408 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:50.020443 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:50.020454 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:50.020563 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:50.020757 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 12:16:50.020830 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:16:50.022250 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:50.015777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42705 (local address 127.7.39.2:37603)
0811 12:16:50.016750 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:50.016788 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:50.018497 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:50.019232 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:50.019241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:50.019338 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:50.020021 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:16:50.020024 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:50.020065 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:16:50.020171 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:50.020545 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:50.020555 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:50.020563 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:16:50.020833 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 12:16:50.021107 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":309,"threads_started":1}
I20260811 12:16:50.023135 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42705
--builtin_ntp_servers=127.7.39.20:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 12:16:50.023804 23416 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42705
I20260811 12:16:50.024088 23416 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:50.024678 23416 heartbeater.cc:511] Master 127.7.39.62:42705 requested a full tablet report, sending...
I20260811 12:16:50.026077 22925 ts_manager.cc:194] Registered new tserver with Master: 6890954f724d411a97f37d9f6df740c5 (127.7.39.2:33545)
W20260811 12:16:50.380802 23422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:50.381258 23422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:50.381414 23422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:50.389593 23422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:50.389714 23422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:50.389876 23422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:50.399585 23422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:50.401026 23422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:50.402987 23422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:50.416697 23430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:50.416736 23431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:50.416759 23433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:50.417634 23422 server_base.cc:1034] running on GCE node
I20260811 12:16:50.418303 23422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:50.419361 23422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:50.420603 23422 hybrid_clock.cc:648] HybridClock initialized: now 1786450610420544 us; error 47 us; skew 500 ppm
I20260811 12:16:50.420971 23422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:16:50.439420 23422 webserver.cc:460] Webserver started at http://127.7.39.3:34459/ using document root <none> and password file <none>
I20260811 12:16:50.440222 23422 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:50.440346 23422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:50.440701 23422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:50.443889 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "b3ee3a3e458941b8989538b2acba6984"
format_stamp: "Formatted at 2026-08-11 12:16:50 on dist-test-slave-rmhg"
server_key: "6f47c0800ae2150fbdf382a4f52ac2bd"
server_key_iv: "baaa89760a3b69032b226d4b0cd705f9"
server_key_version: "encryptionkey@0"
I20260811 12:16:50.444788 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "b3ee3a3e458941b8989538b2acba6984"
format_stamp: "Formatted at 2026-08-11 12:16:50 on dist-test-slave-rmhg"
server_key: "6f47c0800ae2150fbdf382a4f52ac2bd"
server_key_iv: "baaa89760a3b69032b226d4b0cd705f9"
server_key_version: "encryptionkey@0"
I20260811 12:16:50.450532 23422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:16:50.454468 23440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:50.455912 23422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:16:50.456103 23422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "b3ee3a3e458941b8989538b2acba6984"
format_stamp: "Formatted at 2026-08-11 12:16:50 on dist-test-slave-rmhg"
server_key: "6f47c0800ae2150fbdf382a4f52ac2bd"
server_key_iv: "baaa89760a3b69032b226d4b0cd705f9"
server_key_version: "encryptionkey@0"
I20260811 12:16:50.456314 23422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:50.462216 23422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:50.463068 23422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:50.463330 23422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:50.464359 23422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:50.465942 23422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:50.466046 23422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:50.466135 23422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:50.466188 23422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:50.497114 23422 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:44241
I20260811 12:16:50.497182 23629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:44241 every 8 connection(s)
I20260811 12:16:50.498597 23422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:50.505060 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 23422
I20260811 12:16:50.505272 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:16:50.505867 7324 external_mini_cluster.cc:1428] Setting key 456deaaa20c83f2597d9a88edf00e897
I20260811 12:16:50.507616 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:50.501448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:45959 (local address 127.7.39.62:42705)
0811 12:16:50.501641 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:50.501646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:50.503160 (+ 1514us) server_negotiation.cc:407] Connection header received
0811 12:16:50.505087 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:50.505093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:50.505196 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:50.505299 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:50.506856 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:50.506860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:50.506864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:50.506882 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:50.506922 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:50.506938 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:50.507059 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:50.507333 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 12:16:50.507394 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:16:50.509073 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:50.501995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42705 (local address 127.7.39.3:45959)
0811 12:16:50.502969 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:50.503009 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:50.504793 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:50.505664 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:50.505676 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:50.505815 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:50.506509 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:16:50.506512 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:50.506553 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:16:50.506659 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:50.507070 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:50.507093 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:50.507104 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 12:16:50.507470 (+ 366us) client_negotiation.cc:240] Negotiation successful
0811 12:16:50.507767 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":304,"threads_started":1}
I20260811 12:16:50.510418 23630 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42705
I20260811 12:16:50.510754 23630 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:50.511404 23630 heartbeater.cc:511] Master 127.7.39.62:42705 requested a full tablet report, sending...
I20260811 12:16:50.513028 22925 ts_manager.cc:194] Registered new tserver with Master: b3ee3a3e458941b8989538b2acba6984 (127.7.39.3:44241)
I20260811 12:16:50.521777 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:16:50.528364 23216 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:16:50.526807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.7.39.62:42705)
0811 12:16:50.527035 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:50.527043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:50.527444 (+ 401us) server_negotiation.cc:407] Connection header received
0811 12:16:50.527635 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:50.527639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:50.527695 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:50.527782 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:50.528098 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32810: BlockingRecv error: recv got EOF from 127.0.0.1:32810 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:16:50.528965 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 23014
I20260811 12:16:50.532610 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:50.529946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.7.39.1:42197)
0811 12:16:50.530448 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:50.530464 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:50.530491 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:16:50.530552 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:50.530559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:50.530775 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:50.530921 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:50.531898 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:50.531905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:50.531916 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:50.531935 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:50.531977 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:50.531990 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:50.532123 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:50.532239 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 12:16:50.532344 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":235,"threads_started":1}
I20260811 12:16:50.533893 23205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:50.613808 23214 heartbeater.cc:503] Master 127.7.39.62:42705 was elected leader, sending a full tablet report...
I20260811 12:16:50.615309 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 23014
W20260811 12:16:50.639438 22899 connection.cc:535] server connection from 127.7.39.1:53453 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 12:16:50.639779 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 23219
I20260811 12:16:50.643296 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:50.640753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.7.39.2:33545)
0811 12:16:50.641280 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:50.641287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:50.641315 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:16:50.641382 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:50.641388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:50.641679 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:50.641817 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:50.642699 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:50.642705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:50.642718 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:50.642744 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:50.642789 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:50.642805 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:50.642915 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:50.642975 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 12:16:50.643059 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":226,"threads_started":1}
I20260811 12:16:50.644788 23406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:50.726761 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 23219
I20260811 12:16:50.749526 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 23422
I20260811 12:16:50.753748 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:50.750649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.7.39.3:44241)
0811 12:16:50.751298 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:50.751305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:50.751338 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:16:50.751404 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:50.751410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:50.751638 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:50.751784 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:50.752926 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:50.752933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:50.752945 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:16:50.752957 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:50.753005 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:16:50.753019 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:50.753202 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:50.753330 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 12:16:50.753449 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":342,"threads_started":1}
I20260811 12:16:50.755019 23612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:50.838264 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 23422
I20260811 12:16:50.860147 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 22859
I20260811 12:16:50.861423 22974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:50.965653 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 22859
2026-08-11T12:16:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2541 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:16:51 dist-test-slave-rmhg krb5kdc[23663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:16:51 dist-test-slave-rmhg krb5kdc[23663](info): set up 2 sockets
Aug 11 12:16:51 dist-test-slave-rmhg krb5kdc[23663](info): commencing operation
krb5kdc: starting...
W20260811 12:16:52.226209 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:16:52 dist-test-slave-rmhg krb5kdc[23663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:16:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:16:52.536406 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45137
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:45137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:52.886215 23681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:52.886535 23681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:52.886675 23681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:52.894161 23681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:16:52.894287 23681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:52.894340 23681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:52.894397 23681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:16:52.894440 23681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:16:52.904490 23681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:45137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:52.905882 23681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:52.907198 23681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:52.919034 23690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:52.919075 23695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:52.919108 23693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:52.920269 23681 server_base.cc:1034] running on GCE node
I20260811 12:16:52.921070 23681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:52.922606 23681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:52.923826 23681 hybrid_clock.cc:648] HybridClock initialized: now 1786450612923771 us; error 37 us; skew 500 ppm
I20260811 12:16:52.924228 23681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:52 dist-test-slave-rmhg krb5kdc[23663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450612, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:52.927706 23681 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:16:52.929308 23681 webserver.cc:460] Webserver started at http://127.7.39.62:45505/ using document root <none> and password file <none>
I20260811 12:16:52.930032 23681 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:52.930162 23681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:52.930477 23681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:52.933730 23681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "db7ffbf1af424229a77c6aea4b5bbe33"
format_stamp: "Formatted at 2026-08-11 12:16:52 on dist-test-slave-rmhg"
server_key: "ac3c096cf5b2c4fbbbc6e1cb8ba9e8c8"
server_key_iv: "f3d112c27e850bca546d688f3f54a038"
server_key_version: "encryptionkey@0"
I20260811 12:16:52.934523 23681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "db7ffbf1af424229a77c6aea4b5bbe33"
format_stamp: "Formatted at 2026-08-11 12:16:52 on dist-test-slave-rmhg"
server_key: "ac3c096cf5b2c4fbbbc6e1cb8ba9e8c8"
server_key_iv: "f3d112c27e850bca546d688f3f54a038"
server_key_version: "encryptionkey@0"
I20260811 12:16:52.940047 23681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:16:52.943729 23706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:52.945099 23681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 12:16:52.945307 23681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "db7ffbf1af424229a77c6aea4b5bbe33"
format_stamp: "Formatted at 2026-08-11 12:16:52 on dist-test-slave-rmhg"
server_key: "ac3c096cf5b2c4fbbbc6e1cb8ba9e8c8"
server_key_iv: "f3d112c27e850bca546d688f3f54a038"
server_key_version: "encryptionkey@0"
I20260811 12:16:52.945497 23681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:52.973181 23681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:52.977521 23681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:52.977809 23681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:52.995668 23681 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:45137
I20260811 12:16:52.995698 23789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:45137 every 8 connection(s)
I20260811 12:16:52.997181 23681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:16:53.001307 23790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:53.007145 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 23681
I20260811 12:16:53.007419 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:16:53.008251 7324 external_mini_cluster.cc:1428] Setting key 86162346df98eed191eccbe1a183c2e2
I20260811 12:16:53.013832 23790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33: Bootstrap starting.
I20260811 12:16:53.017992 23790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:53.019412 23790 log.cc:824] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:53.023478 23790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33: No bootstrap required, opened a new log
Aug 11 12:16:53 dist-test-slave-rmhg krb5kdc[23663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:53.033238 23790 raft_consensus.cc:348] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7ffbf1af424229a77c6aea4b5bbe33" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45137 } }
I20260811 12:16:53.033715 23790 raft_consensus.cc:374] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:53.033818 23790 raft_consensus.cc:729] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db7ffbf1af424229a77c6aea4b5bbe33, State: Initialized, Role: FOLLOWER
I20260811 12:16:53.034483 23790 consensus_queue.cc:260] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7ffbf1af424229a77c6aea4b5bbe33" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45137 } }
I20260811 12:16:53.034670 23794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:53.010726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46956 (local address 127.7.39.62:45137)
0811 12:16:53.011628 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:53.011641 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:53.011691 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:16:53.013514 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:53.013550 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:53.014028 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:53.014409 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:53.016523 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:53.017994 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:53.019859 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:53.020978 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:53.021147 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:53.026249 (+ 5102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:53.026259 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:53.026280 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:53.026329 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:53.029885 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:53.030871 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:53.030882 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:53.030890 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:53.031104 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:53.031573 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:53.031577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:53.031579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:53.032194 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:53.032456 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:53.032902 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 12:16:53.033355 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":407,"threads_started":1}
I20260811 12:16:53.034758 23790 raft_consensus.cc:388] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:16:53.034914 23790 raft_consensus.cc:482] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:16:53.035128 23790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:53.037990 23790 raft_consensus.cc:504] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7ffbf1af424229a77c6aea4b5bbe33" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45137 } }
I20260811 12:16:53.038456 23790 leader_election.cc:302] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db7ffbf1af424229a77c6aea4b5bbe33; no voters:
I20260811 12:16:53.039005 23790 leader_election.cc:288] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:16:53.039172 23797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:53.039865 23797 raft_consensus.cc:686] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [term 1 LEADER]: Becoming Leader. State: Replica: db7ffbf1af424229a77c6aea4b5bbe33, State: Running, Role: LEADER
I20260811 12:16:53.040445 23797 consensus_queue.cc:237] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7ffbf1af424229a77c6aea4b5bbe33" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45137 } }
I20260811 12:16:53.041378 23790 sys_catalog.cc:564] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:16:53.043313 23800 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db7ffbf1af424229a77c6aea4b5bbe33. Latest consensus state: current_term: 1 leader_uuid: "db7ffbf1af424229a77c6aea4b5bbe33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7ffbf1af424229a77c6aea4b5bbe33" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45137 } } }
I20260811 12:16:53.043779 23800 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:53.043599 23799 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db7ffbf1af424229a77c6aea4b5bbe33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7ffbf1af424229a77c6aea4b5bbe33" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45137 } } }
I20260811 12:16:53.043880 23799 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33 [sys.catalog]: This master's current role is: LEADER
I20260811 12:16:53.046746 23810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:16:53.051510 23810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:16:53.061609 23810 catalog_manager.cc:1347] Generated new cluster ID: 0baeb0a6965448aeb03fccf631cecf6d
I20260811 12:16:53.061736 23810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:16:53.078835 23810 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:16:53.080808 23810 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:16:53.091301 23810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33: Generated new TSK 0
I20260811 12:16:53.092499 23810 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:16:53.163239 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45137
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:53.515448 23828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:53.515761 23828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:53.515893 23828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:53.523550 23828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:53.523689 23828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:53.523847 23828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:16:53.534118 23828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:53.535691 23828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:53.537205 23828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:53.551312 23835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:53.551394 23839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:53.551355 23834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:53.552402 23828 server_base.cc:1034] running on GCE node
I20260811 12:16:53.553071 23828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:53.554141 23828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:53.555364 23828 hybrid_clock.cc:648] HybridClock initialized: now 1786450613555306 us; error 51 us; skew 500 ppm
I20260811 12:16:53.555733 23828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:53 dist-test-slave-rmhg krb5kdc[23663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450613, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:53.559141 23828 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:16:53.560719 23828 webserver.cc:460] Webserver started at http://127.7.39.1:42667/ using document root <none> and password file <none>
I20260811 12:16:53.561484 23828 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:53.561621 23828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:53.561925 23828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:53.565023 23828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "8b327b7363d5480e9b849ef41afb9096"
format_stamp: "Formatted at 2026-08-11 12:16:53 on dist-test-slave-rmhg"
server_key: "181b63a4b8c56622ebccc361ce9ef4c4"
server_key_iv: "1d77872c149baf1a86e6125e3ea1834b"
server_key_version: "encryptionkey@0"
I20260811 12:16:53.565918 23828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "8b327b7363d5480e9b849ef41afb9096"
format_stamp: "Formatted at 2026-08-11 12:16:53 on dist-test-slave-rmhg"
server_key: "181b63a4b8c56622ebccc361ce9ef4c4"
server_key_iv: "1d77872c149baf1a86e6125e3ea1834b"
server_key_version: "encryptionkey@0"
I20260811 12:16:53.571331 23828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:16:53.575073 23858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:53.576638 23828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:16:53.576820 23828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "8b327b7363d5480e9b849ef41afb9096"
format_stamp: "Formatted at 2026-08-11 12:16:53 on dist-test-slave-rmhg"
server_key: "181b63a4b8c56622ebccc361ce9ef4c4"
server_key_iv: "1d77872c149baf1a86e6125e3ea1834b"
server_key_version: "encryptionkey@0"
I20260811 12:16:53.577023 23828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:53.623154 23828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:53.627884 23828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:53.628230 23828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:53.629344 23828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:53.630786 23828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:53.630869 23828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:53.630995 23828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:53.631052 23828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:53.662909 23828 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:45705
I20260811 12:16:53.662955 24051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:45705 every 8 connection(s)
I20260811 12:16:53.664239 23828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:16:53.665827 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 23828
I20260811 12:16:53.666064 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:16:53.666841 7324 external_mini_cluster.cc:1428] Setting key 3231498e92ef4c08c1e6e94be4b4deee
Aug 11 12:16:53 dist-test-slave-rmhg krb5kdc[23663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450613, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:53.686870 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:53.667249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:58905 (local address 127.7.39.62:45137)
0811 12:16:53.667771 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:53.667777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:53.668836 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 12:16:53.671342 (+ 2506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:53.671347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:53.671425 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:53.671523 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:53.673564 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:53.674305 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:53.676428 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:53.676850 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:53.676949 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:53.681145 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:53.681170 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:53.681175 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:53.681220 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:53.683786 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:53.684523 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:53.684529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:53.684532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:53.684702 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:53.685093 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:53.685098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:53.685102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:53.685468 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:53.685621 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:53.686305 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 12:16:53.686525 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":248,"threads_started":1}
I20260811 12:16:53.687728 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:53.667844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45137 (local address 127.7.39.1:58905)
0811 12:16:53.668661 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:53.668713 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:53.670991 (+ 2278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:53.671806 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:53.671818 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:53.672539 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:53.673378 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:53.673404 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:53.674479 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:53.674484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:53.676265 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:53.676280 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:53.677080 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:53.677086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:53.677410 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:53.678136 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:53.678166 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:53.680858 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:53.683918 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:53.683924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:53.683939 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:53.684367 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:53.684792 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:53.684795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:53.684797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:53.684982 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:53.685576 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:53.685582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:53.685920 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 12:16:53.686157 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 12:16:53.686561 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":337,"threads_started":1}
I20260811 12:16:53.689586 24053 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45137
I20260811 12:16:53.689895 24053 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:53.690641 24053 heartbeater.cc:511] Master 127.7.39.62:45137 requested a full tablet report, sending...
I20260811 12:16:53.693209 23730 ts_manager.cc:194] Registered new tserver with Master: 8b327b7363d5480e9b849ef41afb9096 (127.7.39.1:45705)
I20260811 12:16:53.695256 23730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:58905
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:16:53.736055 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45137
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:54.084205 24064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:54.084610 24064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:54.084761 24064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:54.092538 24064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:54.092674 24064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:54.092820 24064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:16:54.103217 24064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:54.104641 24064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:54.106287 24064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:54.119889 24074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:54.119944 24071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:54.119922 24070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:54.120762 24064 server_base.cc:1034] running on GCE node
I20260811 12:16:54.121431 24064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:54.122438 24064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:54.123667 24064 hybrid_clock.cc:648] HybridClock initialized: now 1786450614123598 us; error 58 us; skew 500 ppm
I20260811 12:16:54.124012 24064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:54 dist-test-slave-rmhg krb5kdc[23663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:54.127528 24064 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:16:54.129103 24064 webserver.cc:460] Webserver started at http://127.7.39.2:34451/ using document root <none> and password file <none>
I20260811 12:16:54.129868 24064 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:54.129997 24064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:54.130306 24064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:54.133528 24064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "64addc5865d744e78fae1b2235b91d31"
format_stamp: "Formatted at 2026-08-11 12:16:54 on dist-test-slave-rmhg"
server_key: "d59c2561d7addae9501a31d7abeaac2b"
server_key_iv: "9d2286f6a73f36386f6d29154369649a"
server_key_version: "encryptionkey@0"
I20260811 12:16:54.134375 24064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "64addc5865d744e78fae1b2235b91d31"
format_stamp: "Formatted at 2026-08-11 12:16:54 on dist-test-slave-rmhg"
server_key: "d59c2561d7addae9501a31d7abeaac2b"
server_key_iv: "9d2286f6a73f36386f6d29154369649a"
server_key_version: "encryptionkey@0"
I20260811 12:16:54.140120 24064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 12:16:54.143975 24086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:54.145555 24064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:16:54.145735 24064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "64addc5865d744e78fae1b2235b91d31"
format_stamp: "Formatted at 2026-08-11 12:16:54 on dist-test-slave-rmhg"
server_key: "d59c2561d7addae9501a31d7abeaac2b"
server_key_iv: "9d2286f6a73f36386f6d29154369649a"
server_key_version: "encryptionkey@0"
I20260811 12:16:54.145933 24064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:54.165712 24064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:54.170189 24064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:54.170480 24064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:54.171499 24064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:54.172928 24064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:54.173005 24064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:54.173127 24064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:54.173208 24064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 12:16:54.205200 24064 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:33265
I20260811 12:16:54.205261 24284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:33265 every 8 connection(s)
I20260811 12:16:54.206538 24064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:16:54.216471 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 24064
I20260811 12:16:54.216653 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:16:54.217167 7324 external_mini_cluster.cc:1428] Setting key ffb60f4bfd87f0c37a301bfd81c08601
Aug 11 12:16:54 dist-test-slave-rmhg krb5kdc[23663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:54.227437 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.209098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:40287 (local address 127.7.39.62:45137)
0811 12:16:54.209582 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.209587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.210584 (+ 997us) server_negotiation.cc:407] Connection header received
0811 12:16:54.212456 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.212462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.212556 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.212657 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:54.214351 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.215099 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.216724 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.217214 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.217341 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.221301 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:54.221307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:54.221310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:54.221347 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:54.224015 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:54.224848 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:54.224853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:54.224855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:54.224996 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:54.225507 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:54.225511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:54.225513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:54.225853 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:54.225999 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.226791 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.227037 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":265,"threads_started":1}
I20260811 12:16:54.228570 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.209658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45137 (local address 127.7.39.2:40287)
0811 12:16:54.210407 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:54.210458 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:54.212155 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:54.212930 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:54.212939 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:54.213540 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:54.214147 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.214178 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.215273 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.215277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.216553 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.216570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.217475 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.217483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.217818 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.218483 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:54.218513 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:54.221016 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:54.224191 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:54.224202 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:54.224220 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:54.224712 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:54.225116 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:54.225121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:54.225125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:54.225388 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:54.225987 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:54.225996 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:54.226393 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 12:16:54.226663 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 12:16:54.227272 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":278,"threads_started":1}
I20260811 12:16:54.230250 24287 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45137
I20260811 12:16:54.230569 24287 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:54.231398 24287 heartbeater.cc:511] Master 127.7.39.62:45137 requested a full tablet report, sending...
I20260811 12:16:54.233078 23730 ts_manager.cc:194] Registered new tserver with Master: 64addc5865d744e78fae1b2235b91d31 (127.7.39.2:33265)
I20260811 12:16:54.233959 23730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:40287
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:16:54.282949 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45137
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:16:54.632519 24299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:16:54.632826 24299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:16:54.632951 24299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:16:54.640413 24299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:16:54.640551 24299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:16:54.640667 24299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:16:54.651387 24299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:16:54.652802 24299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:16:54.654273 24299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:16:54.668637 24308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:54.668704 24309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:16:54.668676 24311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:16:54.669589 24299 server_base.cc:1034] running on GCE node
I20260811 12:16:54.670351 24299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:16:54.671420 24299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:16:54.672654 24299 hybrid_clock.cc:648] HybridClock initialized: now 1786450614672592 us; error 50 us; skew 500 ppm
I20260811 12:16:54.672958 24299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:16:54 dist-test-slave-rmhg krb5kdc[23663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:16:54.676326 24299 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:16:54.677911 24299 webserver.cc:460] Webserver started at http://127.7.39.3:33001/ using document root <none> and password file <none>
I20260811 12:16:54.678625 24299 fs_manager.cc:359] Metadata directory not provided
I20260811 12:16:54.678748 24299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:16:54.679059 24299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:16:54.682248 24299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "ae503f18f62c438aa10ba5f4a96231f8"
format_stamp: "Formatted at 2026-08-11 12:16:54 on dist-test-slave-rmhg"
server_key: "a187ec1388fc9f5d88c19b3734b6efd0"
server_key_iv: "2a33997a0083008d16dba5cbf244f83e"
server_key_version: "encryptionkey@0"
I20260811 12:16:54.683082 24299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "ae503f18f62c438aa10ba5f4a96231f8"
format_stamp: "Formatted at 2026-08-11 12:16:54 on dist-test-slave-rmhg"
server_key: "a187ec1388fc9f5d88c19b3734b6efd0"
server_key_iv: "2a33997a0083008d16dba5cbf244f83e"
server_key_version: "encryptionkey@0"
I20260811 12:16:54.688325 24299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:16:54.691885 24320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:54.693370 24299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:16:54.693535 24299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "ae503f18f62c438aa10ba5f4a96231f8"
format_stamp: "Formatted at 2026-08-11 12:16:54 on dist-test-slave-rmhg"
server_key: "a187ec1388fc9f5d88c19b3734b6efd0"
server_key_iv: "2a33997a0083008d16dba5cbf244f83e"
server_key_version: "encryptionkey@0"
I20260811 12:16:54.693747 24299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:16:54.698683 24053 heartbeater.cc:503] Master 127.7.39.62:45137 was elected leader, sending a full tablet report...
I20260811 12:16:54.711758 24299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:16:54.715982 24299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:16:54.716281 24299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:16:54.717362 24299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:16:54.718813 24299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:16:54.718887 24299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:54.719003 24299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:16:54.719053 24299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:16:54.749924 24299 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:43709
I20260811 12:16:54.750046 24511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:43709 every 8 connection(s)
I20260811 12:16:54.751287 24299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:16:54.753906 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 24299
I20260811 12:16:54.754113 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:16:54.754653 7324 external_mini_cluster.cc:1428] Setting key 8badc639a2d6b577a2ebb11d1e9cc5fa
Aug 11 12:16:54 dist-test-slave-rmhg krb5kdc[23663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:16:54.772581 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.753974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:45781 (local address 127.7.39.62:45137)
0811 12:16:54.754491 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.754498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.755549 (+ 1051us) server_negotiation.cc:407] Connection header received
0811 12:16:54.757697 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.757702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.757811 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.757909 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:54.759540 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.760365 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.762118 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.762545 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.762665 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.766460 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:54.766466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:54.766469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:54.766506 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:54.769286 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:54.770091 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:54.770095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:54.770098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:54.770238 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:54.770706 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:54.770709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:54.770711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:54.770999 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:54.771140 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.771933 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.772197 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":236,"threads_started":1}
I20260811 12:16:54.773639 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.754531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45137 (local address 127.7.39.3:45781)
0811 12:16:54.755384 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:54.755434 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:54.757414 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:54.758182 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:54.758191 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:54.758744 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:16:54.759357 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.759375 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.760547 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.760551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.761957 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.761976 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.762805 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.762813 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.763064 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.763760 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:16:54.763785 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:16:54.766193 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:16:54.769442 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:54.769453 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:54.769474 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:54.769951 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:54.770352 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:16:54.770357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:16:54.770361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:16:54.770581 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:16:54.771136 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:16:54.771144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:16:54.771527 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 12:16:54.771791 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 12:16:54.772397 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":348,"threads_started":1}
I20260811 12:16:54.775131 24514 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45137
I20260811 12:16:54.775441 24514 heartbeater.cc:464] Registering TS with master...
I20260811 12:16:54.776227 24514 heartbeater.cc:511] Master 127.7.39.62:45137 requested a full tablet report, sending...
I20260811 12:16:54.777768 23730 ts_manager.cc:194] Registered new tserver with Master: ae503f18f62c438aa10ba5f4a96231f8 (127.7.39.3:43709)
I20260811 12:16:54.778621 23730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:45781
I20260811 12:16:54.782891 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:16:54.800516 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.787721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46958 (local address 127.7.39.62:45137)
0811 12:16:54.787906 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.787911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.788452 (+ 541us) server_negotiation.cc:407] Connection header received
0811 12:16:54.788636 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.788640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.788725 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.788805 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:54.790368 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.791140 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.792488 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.792925 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.793021 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.795337 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:54.795343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:54.795346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:54.795383 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:54.798109 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:54.798808 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:54.798812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:54.798815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:54.798946 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:54.799238 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:54.799241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:54.799243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:54.799594 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:54.799808 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.800081 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.800280 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:16:54.805084 23730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:16:54.812330 23730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:16:54.837996 24535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.825308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.7.39.1:45705)
0811 12:16:54.827032 (+ 1724us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.827041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.827073 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:16:54.827191 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.827199 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.827473 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.827665 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:54.828447 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.830313 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.833574 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.836534 (+ 2960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.836954 (+ 420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.837261 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.837372 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.837574 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1548,"thread_start_us":1414,"threads_started":1}
I20260811 12:16:54.838079 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.825090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:45705 (local address 127.0.0.1:50650)
0811 12:16:54.825971 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:54.826040 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:54.826261 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:54.827706 (+ 1445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:54.827713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:54.827761 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:54.828202 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.828218 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.830497 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.830504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.833313 (+ 2809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.833333 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.836846 (+ 3513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.836854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.837062 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.837132 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 12:16:54.837293 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 12:16:54.837548 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":22,"thread_start_us":773,"threads_started":1}
I20260811 12:16:54.839669 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.825231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:43709 (local address 127.0.0.1:37096)
0811 12:16:54.827735 (+ 2504us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:54.827753 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:54.827874 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:54.828877 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:54.828883 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:54.828910 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:54.829398 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.829411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.831588 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.831594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.834077 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.834096 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.836582 (+ 2486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.836590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.838875 (+ 2285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.838919 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 12:16:54.839015 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:16:54.839181 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2383,"mutex_wait_us":5,"thread_start_us":2231,"threads_started":1}
I20260811 12:16:54.841468 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.825611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.7.39.3:43709)
0811 12:16:54.827411 (+ 1800us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.827418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.828009 (+ 591us) server_negotiation.cc:407] Connection header received
0811 12:16:54.828129 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.828137 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.828369 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.828576 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:54.829626 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.831385 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.834295 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.835954 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.836099 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.836387 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.840888 (+ 4501us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.841084 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1647,"thread_start_us":1488,"threads_started":1}
I20260811 12:16:54.843391 23934 tablet_service.cc:1467] Processing CreateTablet for tablet 38d0becd0cb347d59315b8fc553d1cc7 (DEFAULT_TABLE table=test-table [id=a5674e6eb93c4f3fa75805a69975bec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:54.845117 23934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38d0becd0cb347d59315b8fc553d1cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:54.845538 24412 tablet_service.cc:1467] Processing CreateTablet for tablet 38d0becd0cb347d59315b8fc553d1cc7 (DEFAULT_TABLE table=test-table [id=a5674e6eb93c4f3fa75805a69975bec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:54.847723 24412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38d0becd0cb347d59315b8fc553d1cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:54.847896 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.827814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33265 (local address 127.0.0.1:52068)
0811 12:16:54.838377 (+ 10563us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:54.838401 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:54.838528 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:54.841503 (+ 2975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:54.841508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:54.841539 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:54.841722 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.841735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.843800 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.843806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.845695 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.845714 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.847328 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.847332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.847470 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.847496 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:16:54.847550 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 12:16:54.847649 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10443,"thread_start_us":12378,"threads_started":1}
I20260811 12:16:54.848786 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.828285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52068 (local address 127.7.39.2:33265)
0811 12:16:54.840784 (+ 12499us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.840793 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.840824 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:16:54.840941 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.840948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.841261 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.841955 (+ 694us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:54.842148 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.843610 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.845907 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.847150 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.847968 (+ 818us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.848316 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.848387 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.848515 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12338,"thread_start_us":12212,"threads_started":1}
I20260811 12:16:54.852187 24172 tablet_service.cc:1467] Processing CreateTablet for tablet 38d0becd0cb347d59315b8fc553d1cc7 (DEFAULT_TABLE table=test-table [id=a5674e6eb93c4f3fa75805a69975bec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:16:54.853444 24172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38d0becd0cb347d59315b8fc553d1cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:16:54.861984 24540 tablet_bootstrap.cc:492] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Bootstrap starting.
I20260811 12:16:54.864176 24541 tablet_bootstrap.cc:492] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: Bootstrap starting.
I20260811 12:16:54.866882 24540 tablet_bootstrap.cc:654] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:54.867273 24542 tablet_bootstrap.cc:492] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Bootstrap starting.
I20260811 12:16:54.867654 24541 tablet_bootstrap.cc:654] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:54.868468 24540 log.cc:824] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:54.868911 24541 log.cc:824] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:54.871737 24540 tablet_bootstrap.cc:492] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: No bootstrap required, opened a new log
I20260811 12:16:54.871783 24542 tablet_bootstrap.cc:654] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Neither blocks nor log segments found. Creating new log.
I20260811 12:16:54.872020 24540 ts_tablet_manager.cc:1397] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260811 12:16:54.872541 24541 tablet_bootstrap.cc:492] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: No bootstrap required, opened a new log
I20260811 12:16:54.872824 24541 ts_tablet_manager.cc:1397] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 12:16:54.873010 24542 log.cc:824] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Log is configured to *not* fsync() on all Append() calls
I20260811 12:16:54.876257 24542 tablet_bootstrap.cc:492] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: No bootstrap required, opened a new log
I20260811 12:16:54.876547 24542 ts_tablet_manager.cc:1397] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 12:16:54.880090 24540 raft_consensus.cc:348] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.880481 24540 raft_consensus.cc:374] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:54.880579 24540 raft_consensus.cc:729] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b327b7363d5480e9b849ef41afb9096, State: Initialized, Role: FOLLOWER
I20260811 12:16:54.881286 24540 consensus_queue.cc:260] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.883147 24540 ts_tablet_manager.cc:1428] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Time spent starting tablet: real 0.011s user 0.010s sys 0.002s
I20260811 12:16:54.883203 24541 raft_consensus.cc:348] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.883688 24541 raft_consensus.cc:374] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:54.883795 24541 raft_consensus.cc:729] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae503f18f62c438aa10ba5f4a96231f8, State: Initialized, Role: FOLLOWER
I20260811 12:16:54.884414 24541 consensus_queue.cc:260] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.885610 24514 heartbeater.cc:503] Master 127.7.39.62:45137 was elected leader, sending a full tablet report...
I20260811 12:16:54.886319 24541 ts_tablet_manager.cc:1428] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260811 12:16:54.889904 24542 raft_consensus.cc:348] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.890408 24542 raft_consensus.cc:374] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:16:54.890517 24542 raft_consensus.cc:729] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64addc5865d744e78fae1b2235b91d31, State: Initialized, Role: FOLLOWER
I20260811 12:16:54.891055 24542 consensus_queue.cc:260] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.892060 24287 heartbeater.cc:503] Master 127.7.39.62:45137 was elected leader, sending a full tablet report...
I20260811 12:16:54.892513 24542 ts_tablet_manager.cc:1428] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260811 12:16:54.897722 24548 raft_consensus.cc:482] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:16:54.898066 24548 raft_consensus.cc:504] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.899941 24548 leader_election.cc:288] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae503f18f62c438aa10ba5f4a96231f8 (127.7.39.3:43709), 64addc5865d744e78fae1b2235b91d31 (127.7.39.2:33265)
I20260811 12:16:54.907125 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.901223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:51533 (local address 127.7.39.2:33265)
0811 12:16:54.901446 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.901453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.901615 (+ 162us) server_negotiation.cc:407] Connection header received
0811 12:16:54.901792 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.901797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.901874 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.901976 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:54.902598 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.903382 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.905267 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.906098 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.906247 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.906342 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.906712 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.906853 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:16:54.907114 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.900228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:34487 (local address 127.7.39.3:43709)
0811 12:16:54.900478 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.900484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.900886 (+ 402us) server_negotiation.cc:407] Connection header received
0811 12:16:54.901105 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.901110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.901210 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.901307 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:16:54.902189 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.903089 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.905120 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.906015 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.906126 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.906235 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.906702 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.906846 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:16:54.907274 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.900163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:43709 (local address 127.7.39.1:34487)
0811 12:16:54.900774 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:54.900791 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:54.900944 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:54.901530 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:54.901534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:54.901565 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:54.901976 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.901991 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.903237 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.903242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.904916 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.904932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.906257 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.906263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.906475 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.906552 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 12:16:54.906640 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:16:54.906786 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":30,"spinlock_wait_cycles":6784,"thread_start_us":298,"threads_started":1}
I20260811 12:16:54.907321 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.900866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33265 (local address 127.7.39.1:51533)
0811 12:16:54.901502 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 12:16:54.901528 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:16:54.901664 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:16:54.902009 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:16:54.902012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:16:54.902034 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:16:54.902444 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.902456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.903534 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.903538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.905073 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:16:54.905091 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.906346 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.906351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:16:54.906552 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.906577 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:16:54.906674 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:16:54.906819 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":20,"spinlock_wait_cycles":768,"thread_start_us":418,"threads_started":1}
I20260811 12:16:54.908600 24212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38d0becd0cb347d59315b8fc553d1cc7" candidate_uuid: "8b327b7363d5480e9b849ef41afb9096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64addc5865d744e78fae1b2235b91d31" is_pre_election: true
I20260811 12:16:54.909113 24212 raft_consensus.cc:2454] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b327b7363d5480e9b849ef41afb9096 in term 0.
I20260811 12:16:54.909648 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38d0becd0cb347d59315b8fc553d1cc7" candidate_uuid: "8b327b7363d5480e9b849ef41afb9096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae503f18f62c438aa10ba5f4a96231f8" is_pre_election: true
I20260811 12:16:54.909857 23862 leader_election.cc:302] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64addc5865d744e78fae1b2235b91d31, 8b327b7363d5480e9b849ef41afb9096; no voters:
I20260811 12:16:54.910156 24445 raft_consensus.cc:2454] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b327b7363d5480e9b849ef41afb9096 in term 0.
I20260811 12:16:54.910286 24548 raft_consensus.cc:2781] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:16:54.910429 24548 raft_consensus.cc:482] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:16:54.910526 24548 raft_consensus.cc:3037] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:54.913064 24548 raft_consensus.cc:504] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.913722 24548 leader_election.cc:288] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [CANDIDATE]: Term 1 election: Requested vote from peers ae503f18f62c438aa10ba5f4a96231f8 (127.7.39.3:43709), 64addc5865d744e78fae1b2235b91d31 (127.7.39.2:33265)
I20260811 12:16:54.914201 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38d0becd0cb347d59315b8fc553d1cc7" candidate_uuid: "8b327b7363d5480e9b849ef41afb9096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae503f18f62c438aa10ba5f4a96231f8"
I20260811 12:16:54.914283 24212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38d0becd0cb347d59315b8fc553d1cc7" candidate_uuid: "8b327b7363d5480e9b849ef41afb9096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64addc5865d744e78fae1b2235b91d31"
I20260811 12:16:54.914386 24445 raft_consensus.cc:3037] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:16:54.914507 24212 raft_consensus.cc:3037] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:16:54.916743 24054 tablet.cc:2369] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:16:54.916743 24445 raft_consensus.cc:2454] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b327b7363d5480e9b849ef41afb9096 in term 1.
I20260811 12:16:54.917119 24212 raft_consensus.cc:2454] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b327b7363d5480e9b849ef41afb9096 in term 1.
I20260811 12:16:54.917243 23862 leader_election.cc:302] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b327b7363d5480e9b849ef41afb9096, ae503f18f62c438aa10ba5f4a96231f8; no voters:
I20260811 12:16:54.917549 24548 raft_consensus.cc:2781] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:16:54.918026 24548 raft_consensus.cc:686] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 1 LEADER]: Becoming Leader. State: Replica: 8b327b7363d5480e9b849ef41afb9096, State: Running, Role: LEADER
I20260811 12:16:54.918498 24548 consensus_queue.cc:237] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } }
I20260811 12:16:54.923390 23730 catalog_manager.cc:5495] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b327b7363d5480e9b849ef41afb9096 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "8b327b7363d5480e9b849ef41afb9096" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b327b7363d5480e9b849ef41afb9096" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 } health_report { overall_health: UNKNOWN } } }
I20260811 12:16:54.940009 24535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.934459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.7.39.1:45705)
0811 12:16:54.934661 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.934668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:54.934766 (+ 98us) server_negotiation.cc:407] Connection header received
0811 12:16:54.934997 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:54.935001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:54.935094 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:54.935245 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:16:54.935628 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.936364 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.937806 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:54.938240 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:54.938356 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:54.938871 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:16:54.939404 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:16:54.939546 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:54.939655 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 12:16:54.939746 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:16:54.948645 24212 raft_consensus.cc:1261] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 1 FOLLOWER]: Refusing update from remote peer 8b327b7363d5480e9b849ef41afb9096: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:54.948645 24445 raft_consensus.cc:1261] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 1 FOLLOWER]: Refusing update from remote peer 8b327b7363d5480e9b849ef41afb9096: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:16:54.949788 24558 consensus_queue.cc:1035] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64addc5865d744e78fae1b2235b91d31" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:16:54.950183 24548 consensus_queue.cc:1035] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae503f18f62c438aa10ba5f4a96231f8" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 43709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:16:54.958483 24291 tablet.cc:2369] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:16:54.961561 24562 mvcc.cc:205] Tried to move back new op lower bound from 7317301718818754560 to 7317301718710075392. Current Snapshot: MvccSnapshot[applied={T|T < 7317301718818754560}]
I20260811 12:16:54.964813 24564 mvcc.cc:205] Tried to move back new op lower bound from 7317301718818754560 to 7317301718710075392. Current Snapshot: MvccSnapshot[applied={T|T < 7317301718818754560}]
I20260811 12:16:54.978156 23730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:54.978397 23730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:16:54.982695 23730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db7ffbf1af424229a77c6aea4b5bbe33: Sending DeleteTablet for 3 replicas of tablet 38d0becd0cb347d59315b8fc553d1cc7
I20260811 12:16:54.983970 23934 tablet_service.cc:1514] Processing DeleteTablet for tablet 38d0becd0cb347d59315b8fc553d1cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:54 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:50650
I20260811 12:16:54.984993 24578 tablet_replica.cc:330] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: stopping tablet replica
I20260811 12:16:54.985610 24578 raft_consensus.cc:2229] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:16:54.986415 24578 raft_consensus.cc:2258] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:54.989617 24412 tablet_service.cc:1514] Processing DeleteTablet for tablet 38d0becd0cb347d59315b8fc553d1cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:54 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:37096
I20260811 12:16:54.990079 24578 ts_tablet_manager.cc:1905] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:54.990584 24579 tablet_replica.cc:330] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: stopping tablet replica
I20260811 12:16:54.991034 24579 raft_consensus.cc:2229] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:54.991281 24172 tablet_service.cc:1514] Processing DeleteTablet for tablet 38d0becd0cb347d59315b8fc553d1cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:16:54 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:52068
I20260811 12:16:54.991582 24579 raft_consensus.cc:2258] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:54.994180 24579 ts_tablet_manager.cc:1905] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:54.994529 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 23828
I20260811 12:16:54.998721 24578 ts_tablet_manager.cc:1918] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:54.999034 24578 log.cc:1196] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/38d0becd0cb347d59315b8fc553d1cc7
I20260811 12:16:54.999711 24578 ts_tablet_manager.cc:1939] T 38d0becd0cb347d59315b8fc553d1cc7 P 8b327b7363d5480e9b849ef41afb9096: Deleting consensus metadata
I20260811 12:16:55.001956 23711 catalog_manager.cc:4841] TS 8b327b7363d5480e9b849ef41afb9096 (127.7.39.1:45705): tablet 38d0becd0cb347d59315b8fc553d1cc7 (table test-table [id=a5674e6eb93c4f3fa75805a69975bec3]) successfully deleted
I20260811 12:16:55.005797 24580 tablet_replica.cc:330] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: stopping tablet replica
I20260811 12:16:55.006327 24580 raft_consensus.cc:2229] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:16:55.006923 24580 raft_consensus.cc:2258] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:16:55.016188 24579 ts_tablet_manager.cc:1918] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:55.016403 24579 log.cc:1196] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/38d0becd0cb347d59315b8fc553d1cc7
I20260811 12:16:55.016433 24580 ts_tablet_manager.cc:1905] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:16:55.016992 24579 ts_tablet_manager.cc:1939] T 38d0becd0cb347d59315b8fc553d1cc7 P ae503f18f62c438aa10ba5f4a96231f8: Deleting consensus metadata
I20260811 12:16:55.023728 24580 ts_tablet_manager.cc:1918] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:16:55.023927 24580 log.cc:1196] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/38d0becd0cb347d59315b8fc553d1cc7
I20260811 12:16:55.024459 24580 ts_tablet_manager.cc:1939] T 38d0becd0cb347d59315b8fc553d1cc7 P 64addc5865d744e78fae1b2235b91d31: Deleting consensus metadata
I20260811 12:16:55.027284 23712 catalog_manager.cc:4841] TS ae503f18f62c438aa10ba5f4a96231f8 (127.7.39.3:43709): tablet 38d0becd0cb347d59315b8fc553d1cc7 (table test-table [id=a5674e6eb93c4f3fa75805a69975bec3]) successfully deleted
Aug 11 12:16:55 dist-test-slave-rmhg krb5kdc[23663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:16:55.033293 23712 catalog_manager.cc:4841] TS 64addc5865d744e78fae1b2235b91d31 (127.7.39.2:33265): tablet 38d0becd0cb347d59315b8fc553d1cc7 (table test-table [id=a5674e6eb93c4f3fa75805a69975bec3]) successfully deleted
I20260811 12:16:55.040827 24535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:54.996678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.7.39.1:45705)
0811 12:16:54.996853 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:54.996859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:55.005038 (+ 8179us) server_negotiation.cc:407] Connection header received
0811 12:16:55.007832 (+ 2794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:55.007839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:55.007939 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:55.008031 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:55.013219 (+ 5188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:55.014678 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:55.027335 (+ 12657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:55.028244 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:55.028371 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:55.034020 (+ 5649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:55.034039 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:55.034060 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:55.034113 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:55.037782 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:55.038474 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:55.038483 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:55.038487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:55.038677 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:55.039110 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:55.039115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:55.039119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:55.039605 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:55.039934 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:55.040219 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 12:16:55.040463 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:16:55.041666 24045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:55.151255 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 23828
I20260811 12:16:55.181047 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 24064
Aug 11 12:16:55 dist-test-slave-rmhg krb5kdc[23663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:16:55.196903 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:55.181901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52074 (local address 127.7.39.2:33265)
0811 12:16:55.182105 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:55.182111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:55.182133 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:16:55.182288 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:55.182293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:55.182359 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:55.182442 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:55.183714 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:55.184508 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:55.185964 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:55.186374 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:55.186487 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:55.190857 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:55.190875 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:55.190894 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:55.190942 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:55.194004 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:55.194836 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:55.194844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:55.194847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:55.195001 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:55.195426 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:55.195429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:55.195431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:55.195845 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:55.196091 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:55.196416 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 12:16:55.196602 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:16:55.197860 24274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:55.286679 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 24064
I20260811 12:16:55.312937 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 24299
Aug 11 12:16:55 dist-test-slave-rmhg krb5kdc[23663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:16:55.328447 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:16:55.313768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.7.39.3:43709)
0811 12:16:55.313949 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:16:55.313955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:16:55.313976 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:16:55.314180 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:16:55.314185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:16:55.314267 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:16:55.314353 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:16:55.315612 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:55.316361 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:55.317790 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:16:55.318221 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:16:55.318334 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:16:55.322790 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:16:55.322808 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:16:55.322828 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:16:55.322870 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:16:55.325545 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:55.326224 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:55.326231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:55.326233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:55.326382 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:16:55.326833 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:16:55.326836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:16:55.326838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:16:55.327208 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:16:55.327426 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:16:55.327899 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 12:16:55.328134 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:16:55.329124 24500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:55.418183 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 24299
I20260811 12:16:55.441829 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 23681
I20260811 12:16:55.443106 23784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:16:55.554987 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 23681
2026-08-11T12:16:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4596 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:16:55 dist-test-slave-rmhg krb5kdc[24596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:16:55 dist-test-slave-rmhg krb5kdc[24596](info): set up 2 sockets
Aug 11 12:16:55 dist-test-slave-rmhg krb5kdc[24596](info): commencing operation
krb5kdc: starting...
W20260811 12:16:56.816809 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:16:57 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:16:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:16:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.39.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.39.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/rangeradmin_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/rangeradmin_127.7.39.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.39.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.39.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/rangerlookup_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/rangerlookup_127.7.39.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.39.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.39.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/HTTP_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/HTTP_127.7.39.20@KRBTEST.COM.keytab.
I20260811 12:16:57.242355 7324 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDQu5eT/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/postgres -l logfile start
2026-08-11 12:16:59.765 UTC [24644] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 12:16:59.765 UTC [24644] LOG: listening on IPv4 address "127.7.39.20", port 59903
2026-08-11 12:16:59.769 UTC [24644] LOG: listening on Unix socket "/tmp/.s.PGSQL.59903"
2026-08-11 12:16:59.791 UTC [24647] LOG: database system was shut down at 2026-08-11 12:16:58 UTC
2026-08-11 12:16:59.795 UTC [24644] LOG: database system is ready to accept connections
I20260811 12:17:00.901084 7324 mini_postgres.cc:96] Postgres bound to 59903
2026-08-11 12:17:00.926 UTC [24665] FATAL: database "slave" does not exist
127.7.39.20:59903 - accepting connections
I20260811 12:17:00.927573 7324 mini_ranger.cc:161] Starting Ranger...
I20260811 12:17:00.963078 7324 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 12:17:01.472484 7324 mini_ranger.cc:87] Created ranger Postgres database
I20260811 12:17:01.472613 7324 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-admin
2026-08-11 12:17:01,793 [I] DB FLAVOR :POSTGRES
2026-08-11 12:17:01,793 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:17:01,794 [I] Checking connection..
2026-08-11 12:17:01,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 12:17:02,064 [I] Checking connection passed.
2026-08-11 12:17:02,065 [I] --------- Verifying version history table ---------
2026-08-11 12:17:02,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:17:02,327 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:17:02,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:17:02,580 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:17:02,581 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 12:17:02,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 12:17:02.830 UTC [24756] WARNING: there is no transaction in progress
2026-08-11 12:17:02,837 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 12:17:02,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:17:03,110 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 12:17:03,111 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:17:03,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 12:17:03,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 12:17:03,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-rmhg.c.gcp-upstream.internal','N') ;"
2026-08-11 12:17:03,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:17:04,171 [I] Table x_portal_user does not exist in database ranger
2026-08-11 12:17:04,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:17:04,444 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 12:17:04,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:17:04,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:17:04,980 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 12:17:04,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 12:17:06.009 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.027 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.045 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.061 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.078 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.098 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.159 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.167 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.183 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.195 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.206 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.221 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.230 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.237 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.244 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06.251 UTC [24937] WARNING: there is no transaction in progress
2026-08-11 12:17:06,959 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 12:17:06,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 12:17:07,224 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 12:17:07,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:17:07,468 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 12:17:07,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:17:07,736 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 12:17:07,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:17:07,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:17:08,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-rmhg.c.gcp-upstream.internal';"
2026-08-11 12:17:08,523 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 12:17:08,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 12:17:08,766 [I] DB_PATCHES have already been applied
I20260811 12:17:08.772650 7324 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDQu5eT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 12:17:08.772756 7324 mini_ranger.cc:193] Using host: 127.7.39.20
I20260811 12:17:08.794859 7324 mini_ranger.cc:227] Ranger admin URL: http://127.7.39.20:60889
Aug 11, 2026 12:17:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:17:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:17:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:17:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:17:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:17:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:17:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 12:17:10.093537 7324 mini_ranger.cc:160] Time spent starting Ranger: real 9.166s user 0.009s sys 0.066s
Aug 11, 2026 12:17:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:17:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:17:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:17:30 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:17:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 12:17:32.186782 7324 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.39.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.39.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/rangerkms_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/rangerkms_127.7.39.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.39.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.39.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.39.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/HTTP_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.39.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/HTTP_127.7.39.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 12:17:32 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 12:17:32.439317 7324 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 12:17:32.469399 7324 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 12:17:33.383961 7324 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 12:17:33.384152 7324 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms
I20260811 12:17:33.384305 7324 mini_ranger_kms.cc:227] Using postgres at 127.7.39.20:59903
2026-08-11 12:17:33,947 [I] DB FLAVOR :POSTGRES
2026-08-11 12:17:33,947 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:17:33,947 [I] Checking connection
2026-08-11 12:17:33,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 12:17:34,192 [I] connection success
2026-08-11 12:17:34,192 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 12:17:34,192 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 12:17:34,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 12:17:34,456 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 12:17:34,456 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:17:34,456 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 12:17:34,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:59903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 12:17:34,745 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 12:17:35.275008 7324 mini_ranger_kms.cc:310] Created kms service
I20260811 12:17:35.515434 7324 mini_ranger_kms.cc:326] Created kudu user
I20260811 12:17:35.562912 7324 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 12:17:35 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:17:35.987680 7324 mini_ranger_kms.cc:384] Added ranger policy
I20260811 12:17:35.988091 7324 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 12:17:35.988145 7324 mini_ranger_kms.cc:240] Using host: 127.7.39.20
I20260811 12:17:36.021970 7324 mini_ranger_kms.cc:276] Ranger KMS PID: 25247
I20260811 12:17:36.022149 7324 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.39.20:60249
Aug 11, 2026 12:17:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:17:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:17:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:17:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:17:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:17:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 12:17:38 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450658, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.39.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:17:38.663153 7324 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.224s user 0.162s sys 0.454s
I20260811 12:17:38.663511 7324 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 12:17:38.663731 7324 mini_ranger_kms.cc:401] 127.7.39.20:60249/kms/v1/keys
Aug 11 12:17:38 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450658, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.39.20@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
Aug 11 12:17:39 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450658, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.39.20@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
Aug 11 12:17:41 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:17:41.717278 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:42265
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:42265
--ranger_config_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:60249/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:17:42.333877 25324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:17:42.334353 25324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:17:42.334620 25324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:17:42.344610 25324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:17:42.344882 25324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:17:42.345201 25324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:17:42.345417 25324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:17:42.345630 25324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:17:42.358804 25324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.39.20:60249/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:42265
--ranger_config_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:17:42.360350 25324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:17:42.361950 25324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:17:42.379933 25333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:17:42.380105 25338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:17:42.382059 25331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:17:42.383425 25324 server_base.cc:1034] running on GCE node
I20260811 12:17:42.384979 25324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:17:42.387224 25324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:17:42.388497 25324 hybrid_clock.cc:648] HybridClock initialized: now 1786450662388405 us; error 50 us; skew 500 ppm
I20260811 12:17:42.388967 25324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:17:42 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450662, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:17:42.397322 25324 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:17:42.399485 25324 webserver.cc:460] Webserver started at http://127.7.39.62:43671/ using document root <none> and password file <none>
I20260811 12:17:42.400305 25324 fs_manager.cc:359] Metadata directory not provided
I20260811 12:17:42.400543 25324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:17:42.400988 25324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:17:42 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450662, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:17:42.467206 25324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "44eabbde112d471c8a69c3a726aea20b"
format_stamp: "Formatted at 2026-08-11 12:17:42 on dist-test-slave-rmhg"
server_key: "1671ac5bd0ec20e9a1b807fb7d752941"
server_key_iv: "2633f445b357c7eb84e136bff820f54f"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:42.468358 25324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "44eabbde112d471c8a69c3a726aea20b"
format_stamp: "Formatted at 2026-08-11 12:17:42 on dist-test-slave-rmhg"
server_key: "1671ac5bd0ec20e9a1b807fb7d752941"
server_key_iv: "2633f445b357c7eb84e136bff820f54f"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:42.475636 25324 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.001s
I20260811 12:17:42.496783 25353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:42.498816 25324 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 12:17:42.499152 25324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "44eabbde112d471c8a69c3a726aea20b"
format_stamp: "Formatted at 2026-08-11 12:17:42 on dist-test-slave-rmhg"
server_key: "1671ac5bd0ec20e9a1b807fb7d752941"
server_key_iv: "2633f445b357c7eb84e136bff820f54f"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:42.499500 25324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:17:42.526881 25324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:17:42.532135 25324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:17:42.532634 25324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:17:42.577740 25324 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:42265
I20260811 12:17:42.577843 25442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:42265 every 8 connection(s)
I20260811 12:17:42.579741 25324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:17:42.580408 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 25324
I20260811 12:17:42.580636 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:17:42.581419 7324 external_mini_cluster.cc:1428] Setting key 3c5b8671fac60ac38b922dd1575f036b
I20260811 12:17:42.586853 25444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:17:42 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:17:42.604298 25444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b: Bootstrap starting.
I20260811 12:17:42.609702 25444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b: Neither blocks nor log segments found. Creating new log.
I20260811 12:17:42.610752 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:42.584212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41268 (local address 127.7.39.62:42265)
0811 12:17:42.585857 (+ 1645us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:42.585872 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:42.585947 (+ 75us) server_negotiation.cc:407] Connection header received
0811 12:17:42.588729 (+ 2782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:42.588759 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:42.589275 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:42.589684 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:17:42.591486 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:42.593644 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:42.595682 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:42.596721 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:42.596890 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:42.601680 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:17:42.601693 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:17:42.601711 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:17:42.601756 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:17:42.605802 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:42.606591 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:42.606599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:42.606607 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:42.606888 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:42.607388 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:42.607393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:42.607395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:42.608227 (+ 832us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:17:42.608687 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:42.609110 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 12:17:42.609591 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1369,"thread_start_us":1065,"threads_started":1}
I20260811 12:17:42.611292 25444 log.cc:824] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:17:42.616004 25444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b: No bootstrap required, opened a new log
I20260811 12:17:42.628507 25444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eabbde112d471c8a69c3a726aea20b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42265 } }
I20260811 12:17:42.629035 25444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:17:42.629277 25444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44eabbde112d471c8a69c3a726aea20b, State: Initialized, Role: FOLLOWER
I20260811 12:17:42.630205 25444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eabbde112d471c8a69c3a726aea20b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42265 } }
I20260811 12:17:42.630697 25444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:17:42.631031 25444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:17:42.631294 25444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:17:42.634500 25444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eabbde112d471c8a69c3a726aea20b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42265 } }
I20260811 12:17:42.635111 25444 leader_election.cc:302] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44eabbde112d471c8a69c3a726aea20b; no voters:
I20260811 12:17:42.636050 25444 leader_election.cc:288] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:17:42.636235 25451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:17:42.639410 25444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:17:42.642246 25451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [term 1 LEADER]: Becoming Leader. State: Replica: 44eabbde112d471c8a69c3a726aea20b, State: Running, Role: LEADER
I20260811 12:17:42.643349 25451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eabbde112d471c8a69c3a726aea20b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42265 } }
I20260811 12:17:42.650007 25454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44eabbde112d471c8a69c3a726aea20b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eabbde112d471c8a69c3a726aea20b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42265 } } }
I20260811 12:17:42.649823 25456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44eabbde112d471c8a69c3a726aea20b. Latest consensus state: current_term: 1 leader_uuid: "44eabbde112d471c8a69c3a726aea20b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eabbde112d471c8a69c3a726aea20b" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 42265 } } }
I20260811 12:17:42.650841 25456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [sys.catalog]: This master's current role is: LEADER
I20260811 12:17:42.650880 25454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b [sys.catalog]: This master's current role is: LEADER
I20260811 12:17:42.654233 25472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:17:42.662238 25472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:17:42.663549 25444 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:17:42.685900 25472 catalog_manager.cc:1347] Generated new cluster ID: e8a76d91b989489ca88a1e79fe7c2877
I20260811 12:17:42.686074 25472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:17:42.712169 25472 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:17:42.714203 25472 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:17:42.726621 25472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b: Generated new TSK 0
I20260811 12:17:42.727571 25472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:17:43.650200 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:60249/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42265
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:17:43 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 12:17:44.253019 25514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:17:44.253506 25514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:17:44.253724 25514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:17:44.265890 25514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:17:44.266142 25514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:17:44.266420 25514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:17:44.282333 25514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.39.20:60249/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:17:44.284307 25514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:17:44.286031 25514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:17:44.307194 25529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:17:44.307101 25532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:17:44.307555 25535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:17:44.323396 25514 server_base.cc:1034] running on GCE node
I20260811 12:17:44.324378 25514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:17:44.325956 25514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:17:44.327349 25514 hybrid_clock.cc:648] HybridClock initialized: now 1786450664327197 us; error 67 us; skew 500 ppm
I20260811 12:17:44.327919 25514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:17:44 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450664, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:17:44.333276 25514 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:17:44.335911 25514 webserver.cc:460] Webserver started at http://127.7.39.1:36791/ using document root <none> and password file <none>
I20260811 12:17:44.336869 25514 fs_manager.cc:359] Metadata directory not provided
I20260811 12:17:44.337100 25514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:17:44.337585 25514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:17:44 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450664, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:17:44.360518 25514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "e4121625b9094e6d8a7199e2a05b0422"
format_stamp: "Formatted at 2026-08-11 12:17:44 on dist-test-slave-rmhg"
server_key: "751a1193194f8fa166f39956f3f7ed51"
server_key_iv: "b97d1e980695c5acd467a822810fb706"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:44.362758 25514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "e4121625b9094e6d8a7199e2a05b0422"
format_stamp: "Formatted at 2026-08-11 12:17:44 on dist-test-slave-rmhg"
server_key: "751a1193194f8fa166f39956f3f7ed51"
server_key_iv: "b97d1e980695c5acd467a822810fb706"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:44.397101 25514 fs_manager.cc:691] Time spent creating directory manager: real 0.034s user 0.017s sys 0.012s
I20260811 12:17:44.438854 25544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:44.441704 25514 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.009s sys 0.004s
I20260811 12:17:44.441977 25514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "e4121625b9094e6d8a7199e2a05b0422"
format_stamp: "Formatted at 2026-08-11 12:17:44 on dist-test-slave-rmhg"
server_key: "751a1193194f8fa166f39956f3f7ed51"
server_key_iv: "b97d1e980695c5acd467a822810fb706"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:44.442324 25514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:17:44.517611 25514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:17:44.525456 25514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:17:44.525947 25514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:17:44.533548 25514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:17:44.536978 25514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:17:44.537122 25514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:44.537415 25514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:17:44.537570 25514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:17:44 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:17:44.690567 25514 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:46567
I20260811 12:17:44.690634 25741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:46567 every 8 connection(s)
I20260811 12:17:44.692878 25514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:17:44.695175 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 25514
I20260811 12:17:44.695410 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:17:44.696298 7324 external_mini_cluster.cc:1428] Setting key 5f303bb93365a58b4cd9b37cd9ddc77b
Aug 11 12:17:44 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450664, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:17:44.744418 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:44.705422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:33881 (local address 127.7.39.62:42265)
0811 12:17:44.720481 (+ 15059us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:44.720488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:44.720519 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:17:44.720591 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:44.720596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:44.720714 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:44.722418 (+ 1704us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:17:44.725091 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:44.726771 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:44.730230 (+ 3459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:44.731399 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:44.731546 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:44.737821 (+ 6275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:17:44.737827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:17:44.737831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:17:44.737881 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:17:44.740867 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:44.741794 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:44.741798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:44.741801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:44.741950 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:44.742497 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:44.742501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:44.742503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:44.742812 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:17:44.742965 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:44.743781 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 12:17:44.744039 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14878,"thread_start_us":15404,"threads_started":1}
I20260811 12:17:44.745828 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:44.698292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42265 (local address 127.7.39.1:33881)
0811 12:17:44.707432 (+ 9140us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:44.707537 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:44.712629 (+ 5092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:44.721822 (+ 9193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:44.721839 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:44.723342 (+ 1503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:17:44.724767 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:44.724805 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:44.727043 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:44.727049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:44.729994 (+ 2945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:44.730017 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:44.732081 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:44.732093 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:44.732508 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:44.733651 (+ 1143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:17:44.733684 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:17:44.737485 (+ 3801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:17:44.741051 (+ 3566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:17:44.741063 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:17:44.741082 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:17:44.741635 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:17:44.742074 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:17:44.742079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:17:44.742083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:17:44.742305 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:17:44.742971 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:17:44.742980 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:17:44.743350 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 12:17:44.743659 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 12:17:44.744267 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8580,"thread_start_us":10751,"threads_started":1}
I20260811 12:17:44.747612 25742 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42265
I20260811 12:17:44.747936 25742 heartbeater.cc:464] Registering TS with master...
I20260811 12:17:44.748675 25742 heartbeater.cc:511] Master 127.7.39.62:42265 requested a full tablet report, sending...
I20260811 12:17:44.751317 25390 ts_manager.cc:194] Registered new tserver with Master: e4121625b9094e6d8a7199e2a05b0422 (127.7.39.1:46567)
I20260811 12:17:44.753317 25390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:33881
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:17:44.806433 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:60249/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42265
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:17:44 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
W20260811 12:17:45.599941 25749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:17:45.600399 25749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:17:45.600663 25749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:17:45.611775 25749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:17:45.612002 25749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:17:45.612258 25749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:17:45.625864 25749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.39.20:60249/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:17:45.627470 25749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:17:45.629045 25749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:17:45.648267 25780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:17:45.648337 25778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:17:45.648154 25783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:17:45.649190 25749 server_base.cc:1034] running on GCE node
I20260811 12:17:45.650050 25749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:17:45.651378 25749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:17:45.652694 25749 hybrid_clock.cc:648] HybridClock initialized: now 1786450665652580 us; error 52 us; skew 500 ppm
I20260811 12:17:45.653285 25749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:17:45 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450665, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:17:45.657464 25749 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:17:45.659358 25749 webserver.cc:460] Webserver started at http://127.7.39.2:35807/ using document root <none> and password file <none>
I20260811 12:17:45.660147 25749 fs_manager.cc:359] Metadata directory not provided
I20260811 12:17:45.660305 25749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:17:45.660642 25749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:17:45 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450665, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:17:45.680466 25749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "356e375ace3c438ca34c15fa9997fe2a"
format_stamp: "Formatted at 2026-08-11 12:17:45 on dist-test-slave-rmhg"
server_key: "6344c21b8bfba25e362f3468cb8b8266"
server_key_iv: "8ccccb442e55e9372691b571606cab7f"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:45.681931 25749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "356e375ace3c438ca34c15fa9997fe2a"
format_stamp: "Formatted at 2026-08-11 12:17:45 on dist-test-slave-rmhg"
server_key: "6344c21b8bfba25e362f3468cb8b8266"
server_key_iv: "8ccccb442e55e9372691b571606cab7f"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:45.690357 25749 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s
I20260811 12:17:45.711318 25792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:45.713660 25749 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:17:45.713891 25749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "356e375ace3c438ca34c15fa9997fe2a"
format_stamp: "Formatted at 2026-08-11 12:17:45 on dist-test-slave-rmhg"
server_key: "6344c21b8bfba25e362f3468cb8b8266"
server_key_iv: "8ccccb442e55e9372691b571606cab7f"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:45.714259 25749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:17:45.736711 25749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:17:45.744625 25749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:17:45.745352 25749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:17:45.747424 25749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:17:45.750720 25749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:17:45.750900 25749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:45.751104 25749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:17:45.751183 25749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:45.756686 25742 heartbeater.cc:503] Master 127.7.39.62:42265 was elected leader, sending a full tablet report...
I20260811 12:17:45.873828 25749 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:33177
I20260811 12:17:45.873910 25975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:33177 every 8 connection(s)
I20260811 12:17:45.875677 25749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:17:45.885300 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 25749
I20260811 12:17:45.885540 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:17:45.886245 7324 external_mini_cluster.cc:1428] Setting key 496ee831a1d188741c051e42e1a1a84c
Aug 11 12:17:45 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450665, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
I20260811 12:17:45.953176 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:45.881381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:51373 (local address 127.7.39.62:42265)
0811 12:17:45.919088 (+ 37707us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:45.919098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:45.921581 (+ 2483us) server_negotiation.cc:407] Connection header received
0811 12:17:45.929073 (+ 7492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:45.929080 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:45.929230 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:45.929337 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:17:45.932208 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:45.933680 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:45.936617 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:45.937413 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:45.937517 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:45.943644 (+ 6127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:17:45.943654 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:17:45.943659 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:17:45.943742 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:17:45.948092 (+ 4350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:45.949551 (+ 1459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:45.949557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:45.949560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:45.949822 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:45.950620 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:45.950626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:45.950629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:45.951092 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:17:45.951222 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:45.952397 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0811 12:17:45.952646 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37484,"thread_start_us":37731,"threads_started":1}
I20260811 12:17:45.955590 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:45.880520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42265 (local address 127.7.39.2:51373)
0811 12:17:45.920269 (+ 39749us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:45.920489 (+ 220us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:45.928707 (+ 8218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:45.929894 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:45.929908 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:45.930737 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:17:45.931938 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:45.931981 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:45.933942 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:45.933948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:45.936429 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:45.936447 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:45.937939 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:45.937947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:45.938302 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:45.939297 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:17:45.939334 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:17:45.943234 (+ 3900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:17:45.948473 (+ 5239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:17:45.948485 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:17:45.948509 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:17:45.949382 (+ 873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:17:45.950051 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:17:45.950056 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:17:45.950060 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:17:45.950461 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:17:45.951345 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:17:45.951354 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:17:45.951880 (+ 526us) client_negotiation.cc:769] Sending connection context
0811 12:17:45.953472 (+ 1592us) client_negotiation.cc:240] Negotiation successful
0811 12:17:45.954135 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39297,"thread_start_us":44235,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
I20260811 12:17:45.964955 25976 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42265
I20260811 12:17:45.965405 25976 heartbeater.cc:464] Registering TS with master...
I20260811 12:17:45.966416 25976 heartbeater.cc:511] Master 127.7.39.62:42265 requested a full tablet report, sending...
I20260811 12:17:45.968319 25390 ts_manager.cc:194] Registered new tserver with Master: 356e375ace3c438ca34c15fa9997fe2a (127.7.39.2:33177)
I20260811 12:17:45.969470 25390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:51373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
W20260811 12:17:45.994274 7324 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.39.3: real 0.104s user 0.004s sys 0.023s
I20260811 12:17:45.995357 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:60249/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42265
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:17:46.616053 25984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:17:46.616492 25984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:17:46.616708 25984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:17:46.627735 25984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:17:46.627934 25984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:17:46.628209 25984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:17:46.641492 25984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.39.20:60249/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:17:46.643200 25984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:17:46.644820 25984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:17:46.664963 25993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:17:46.665025 25994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:17:46.665246 25996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:17:46.666788 25984 server_base.cc:1034] running on GCE node
I20260811 12:17:46.667495 25984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:17:46.668815 25984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:17:46.670045 25984 hybrid_clock.cc:648] HybridClock initialized: now 1786450666669970 us; error 63 us; skew 500 ppm
I20260811 12:17:46.670507 25984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:17:46 dist-test-slave-rmhg krb5kdc[24596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450666, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:17:46.675244 25984 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:17:46.677632 25984 webserver.cc:460] Webserver started at http://127.7.39.3:45021/ using document root <none> and password file <none>
I20260811 12:17:46.678506 25984 fs_manager.cc:359] Metadata directory not provided
I20260811 12:17:46.678686 25984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:17:46.679178 25984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:17:46 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450666, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:17:46.698043 25984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "fde09377f0ce4ccf8634080cfa49d7a6"
format_stamp: "Formatted at 2026-08-11 12:17:46 on dist-test-slave-rmhg"
server_key: "dd046fadb619cb2e4b01c8f1d6ccad27"
server_key_iv: "475ca2bc2afd702852bd6706bc74a155"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:46.699460 25984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "fde09377f0ce4ccf8634080cfa49d7a6"
format_stamp: "Formatted at 2026-08-11 12:17:46 on dist-test-slave-rmhg"
server_key: "dd046fadb619cb2e4b01c8f1d6ccad27"
server_key_iv: "475ca2bc2afd702852bd6706bc74a155"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:46.707598 25984 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.008s
I20260811 12:17:46.725765 26007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:46.727659 25984 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 12:17:46.727975 25984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "fde09377f0ce4ccf8634080cfa49d7a6"
format_stamp: "Formatted at 2026-08-11 12:17:46 on dist-test-slave-rmhg"
server_key: "dd046fadb619cb2e4b01c8f1d6ccad27"
server_key_iv: "475ca2bc2afd702852bd6706bc74a155"
server_key_version: "kuduclusterkey@0"
I20260811 12:17:46.728300 25984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:17:46.758508 25984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:17:46.764806 25984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:17:46.765300 25984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:17:46.766702 25984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:17:46.768934 25984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:17:46.769075 25984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:46.769269 25984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:17:46.769404 25984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:17:46.844101 25984 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:44575
I20260811 12:17:46.844213 26193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:44575 every 8 connection(s)
I20260811 12:17:46.845872 25984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:17:46.853753 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 25984
I20260811 12:17:46.853935 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:17:46.854661 7324 external_mini_cluster.cc:1428] Setting key f72e45879c33e104612be2dbfce6870d
Aug 11 12:17:46 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450666, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:17:46.872428 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.848846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:36055 (local address 127.7.39.62:42265)
0811 12:17:46.849700 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:46.849707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:46.851199 (+ 1492us) server_negotiation.cc:407] Connection header received
0811 12:17:46.853637 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:46.853642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:46.853761 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:46.853871 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:17:46.856665 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.858006 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.859928 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.860583 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.860700 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.865252 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:17:46.865257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:17:46.865261 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:17:46.865296 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:17:46.868697 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:46.869587 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:46.869591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:46.869594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:46.869792 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:46.870265 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:46.870268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:46.870270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:46.870669 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:17:46.870879 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:46.871571 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 12:17:46.871807 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":482,"threads_started":1}
I20260811 12:17:46.873710 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.849704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:42265 (local address 127.7.39.3:36055)
0811 12:17:46.851028 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:46.851079 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:46.853286 (+ 2207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:46.855079 (+ 1793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:46.855090 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:46.855628 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:17:46.856492 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:46.856512 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.858152 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.858156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:46.859744 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:46.859758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.860930 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.860941 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:46.861304 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.862063 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:17:46.862088 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:17:46.865007 (+ 2919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:17:46.868857 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:17:46.868866 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:17:46.868883 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:17:46.869493 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:17:46.869909 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:17:46.869913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:17:46.869915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:17:46.870137 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:17:46.870817 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:17:46.870826 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:17:46.871194 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 12:17:46.871465 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 12:17:46.872068 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":611,"threads_started":1}
I20260811 12:17:46.875500 26194 heartbeater.cc:347] Connected to a master server at 127.7.39.62:42265
I20260811 12:17:46.875957 26194 heartbeater.cc:464] Registering TS with master...
I20260811 12:17:46.877074 26194 heartbeater.cc:511] Master 127.7.39.62:42265 requested a full tablet report, sending...
I20260811 12:17:46.878934 25390 ts_manager.cc:194] Registered new tserver with Master: fde09377f0ce4ccf8634080cfa49d7a6 (127.7.39.3:44575)
I20260811 12:17:46.879808 25390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:36055
I20260811 12:17:46.883460 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:17:46.907033 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.889107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.7.39.62:42265)
0811 12:17:46.889322 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:46.889327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:46.890236 (+ 909us) server_negotiation.cc:407] Connection header received
0811 12:17:46.890460 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:46.890466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:46.890547 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:46.890631 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:17:46.892885 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.894001 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.895823 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.896460 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.896584 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.899746 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:17:46.899752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:17:46.899756 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:17:46.899797 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:17:46.903383 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:46.904332 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:46.904337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:46.904339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:46.904541 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:46.905200 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:46.905204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:46.905206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:46.905580 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:17:46.905918 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:46.906322 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 12:17:46.906654 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 12:17:46.911746 25390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:17:46.922648 25390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:17:46.955202 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.939747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.7.39.1:46567)
0811 12:17:46.941746 (+ 1999us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:46.941756 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:46.942665 (+ 909us) server_negotiation.cc:407] Connection header received
0811 12:17:46.942774 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:46.942782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:46.943081 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:46.943330 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:17:46.945882 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.948219 (+ 2337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.952049 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.953496 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.953631 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.953955 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:46.954700 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 12:17:46.954918 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1772,"thread_start_us":1587,"threads_started":1}
I20260811 12:17:46.955839 26216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.939517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:46567 (local address 127.0.0.1:34222)
0811 12:17:46.942158 (+ 2641us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:46.942217 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:46.942438 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:46.943897 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:46.943906 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:46.943962 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:17:46.945607 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:46.945624 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.948407 (+ 2783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.948413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:46.951888 (+ 3475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:46.951905 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.953697 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.953704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:46.953886 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.953949 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 12:17:46.954085 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 12:17:46.954290 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2298,"mutex_wait_us":20,"thread_start_us":1079,"threads_started":1}
I20260811 12:17:46.957778 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.942428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:44575 (local address 127.0.0.1:59908)
0811 12:17:46.944518 (+ 2090us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:46.944542 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:46.944708 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:46.945955 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:46.945959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:46.945984 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:17:46.947169 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:46.947185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.949467 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.949473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:46.953473 (+ 4000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:46.954090 (+ 617us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.956708 (+ 2618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.956717 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:46.956909 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.956943 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:17:46.957043 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 12:17:46.957206 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1937,"mutex_wait_us":44,"thread_start_us":1708,"threads_started":1}
I20260811 12:17:46.957866 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.941105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.7.39.2:33177)
0811 12:17:46.942305 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:46.942314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:46.944292 (+ 1978us) server_negotiation.cc:407] Connection header received
0811 12:17:46.944675 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:46.944684 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:46.944979 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:46.945239 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:17:46.946981 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.948961 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.953721 (+ 4760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.955735 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.955843 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.956180 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:46.957261 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0811 12:17:46.957461 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":960,"thread_start_us":1714,"threads_started":1}
I20260811 12:17:46.957978 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.943296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.7.39.3:44575)
0811 12:17:46.945205 (+ 1909us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:46.945213 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:46.945287 (+ 74us) server_negotiation.cc:407] Connection header received
0811 12:17:46.945388 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:46.945395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:46.945684 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:46.946823 (+ 1139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:17:46.947364 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.949184 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.954261 (+ 5077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.956218 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.956406 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.956719 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:46.957124 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 12:17:46.957341 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1717,"thread_start_us":1541,"threads_started":1}
I20260811 12:17:46.958882 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:46.940383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33177 (local address 127.0.0.1:45034)
0811 12:17:46.944176 (+ 3793us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:46.944196 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:46.944415 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:46.945743 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:46.945747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:46.945774 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:17:46.946681 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:46.946699 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.949171 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.949176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:46.953525 (+ 4349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:46.953538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:46.956708 (+ 3170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:46.956718 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:46.956985 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:46.957011 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:17:46.958328 (+ 1317us) client_negotiation.cc:240] Negotiation successful
0811 12:17:46.958440 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2187,"mutex_wait_us":139,"thread_start_us":2490,"threads_started":1}
I20260811 12:17:46.960762 25634 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe3849ede1542b2a504ba638a87f0a5 (DEFAULT_TABLE table=test-table [id=04b7fccc5b754f03a21ecde8f81a1dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:17:46.962834 25634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe3849ede1542b2a504ba638a87f0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:17:46.963392 26084 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe3849ede1542b2a504ba638a87f0a5 (DEFAULT_TABLE table=test-table [id=04b7fccc5b754f03a21ecde8f81a1dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:17:46.963447 25875 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe3849ede1542b2a504ba638a87f0a5 (DEFAULT_TABLE table=test-table [id=04b7fccc5b754f03a21ecde8f81a1dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:17:46.965070 26084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe3849ede1542b2a504ba638a87f0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:17:46.965540 25875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe3849ede1542b2a504ba638a87f0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:17:46.972496 25976 heartbeater.cc:503] Master 127.7.39.62:42265 was elected leader, sending a full tablet report...
I20260811 12:17:46.981016 26226 tablet_bootstrap.cc:492] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Bootstrap starting.
I20260811 12:17:46.987080 26226 tablet_bootstrap.cc:654] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Neither blocks nor log segments found. Creating new log.
I20260811 12:17:46.987576 26229 tablet_bootstrap.cc:492] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Bootstrap starting.
I20260811 12:17:46.987907 26228 tablet_bootstrap.cc:492] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Bootstrap starting.
I20260811 12:17:46.989459 26226 log.cc:824] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Log is configured to *not* fsync() on all Append() calls
I20260811 12:17:46.992538 26228 tablet_bootstrap.cc:654] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Neither blocks nor log segments found. Creating new log.
I20260811 12:17:46.993369 26229 tablet_bootstrap.cc:654] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Neither blocks nor log segments found. Creating new log.
I20260811 12:17:46.994168 26226 tablet_bootstrap.cc:492] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: No bootstrap required, opened a new log
I20260811 12:17:46.994756 26228 log.cc:824] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:17:46.994856 26226 ts_tablet_manager.cc:1397] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260811 12:17:46.995234 26229 log.cc:824] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:17:46.998965 26228 tablet_bootstrap.cc:492] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: No bootstrap required, opened a new log
I20260811 12:17:46.999383 26228 ts_tablet_manager.cc:1397] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260811 12:17:47.004143 26229 tablet_bootstrap.cc:492] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: No bootstrap required, opened a new log
I20260811 12:17:47.004535 26229 ts_tablet_manager.cc:1397] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.009s
I20260811 12:17:47.006374 26226 raft_consensus.cc:348] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.007179 26226 raft_consensus.cc:374] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:17:47.007378 26226 raft_consensus.cc:729] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4121625b9094e6d8a7199e2a05b0422, State: Initialized, Role: FOLLOWER
I20260811 12:17:47.008272 26226 consensus_queue.cc:260] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.010110 26228 raft_consensus.cc:348] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.010496 26228 raft_consensus.cc:374] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:17:47.010701 26228 raft_consensus.cc:729] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fde09377f0ce4ccf8634080cfa49d7a6, State: Initialized, Role: FOLLOWER
I20260811 12:17:47.011653 26228 consensus_queue.cc:260] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.012391 26226 ts_tablet_manager.cc:1428] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Time spent starting tablet: real 0.017s user 0.008s sys 0.009s
I20260811 12:17:47.013832 26194 heartbeater.cc:503] Master 127.7.39.62:42265 was elected leader, sending a full tablet report...
I20260811 12:17:47.014547 26228 ts_tablet_manager.cc:1428] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Time spent starting tablet: real 0.015s user 0.008s sys 0.009s
I20260811 12:17:47.018814 26229 raft_consensus.cc:348] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.019424 26229 raft_consensus.cc:374] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:17:47.019594 26229 raft_consensus.cc:729] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 356e375ace3c438ca34c15fa9997fe2a, State: Initialized, Role: FOLLOWER
I20260811 12:17:47.020331 26229 consensus_queue.cc:260] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.022109 26229 ts_tablet_manager.cc:1428] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
W20260811 12:17:47.098318 26196 tablet.cc:2369] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:17:47.179731 25979 tablet.cc:2369] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:17:47.214174 26245 raft_consensus.cc:482] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:17:47.214630 26245 raft_consensus.cc:504] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.216804 26245 leader_election.cc:288] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4121625b9094e6d8a7199e2a05b0422 (127.7.39.1:46567), fde09377f0ce4ccf8634080cfa49d7a6 (127.7.39.3:44575)
W20260811 12:17:47.216914 25743 tablet.cc:2369] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:17:47.227528 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.218744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:56761 (local address 127.7.39.3:44575)
0811 12:17:47.218948 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:47.218954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:47.219607 (+ 653us) server_negotiation.cc:407] Connection header received
0811 12:17:47.219784 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:47.219790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:47.219872 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:47.219978 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:17:47.220983 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.222149 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.225088 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.226143 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.226273 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.226426 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:47.226969 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 12:17:47.227103 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:17:47.227522 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.218235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:40801 (local address 127.7.39.1:46567)
0811 12:17:47.218524 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:47.218531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:47.218631 (+ 100us) server_negotiation.cc:407] Connection header received
0811 12:17:47.219070 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:47.219075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:47.219141 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:47.219197 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:17:47.220824 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.221829 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.224890 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.226294 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.226467 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.226611 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:47.227072 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 12:17:47.227208 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 12:17:47.227725 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.216901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:46567 (local address 127.7.39.2:40801)
0811 12:17:47.218503 (+ 1602us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:47.218526 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:47.218811 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:47.219463 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:47.219466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:47.219493 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:17:47.220617 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:47.220635 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.222012 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.222016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:47.224702 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:47.224719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.226681 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.226687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:47.226903 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.226928 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:17:47.227021 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:17:47.227159 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1394,"mutex_wait_us":1,"thread_start_us":973,"threads_started":1}
I20260811 12:17:47.227854 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.218390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:44575 (local address 127.7.39.2:56761)
0811 12:17:47.219488 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:47.219505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:47.219651 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:47.220012 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:47.220016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:47.220035 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:17:47.220815 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:47.220827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.222285 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.222291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:47.224897 (+ 2606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:47.224913 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.226555 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.226562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:47.226795 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.226825 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:17:47.226918 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:17:47.227078 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"thread_start_us":650,"threads_started":1}
I20260811 12:17:47.229373 26114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe3849ede1542b2a504ba638a87f0a5" candidate_uuid: "356e375ace3c438ca34c15fa9997fe2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" is_pre_election: true
I20260811 12:17:47.229435 25666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe3849ede1542b2a504ba638a87f0a5" candidate_uuid: "356e375ace3c438ca34c15fa9997fe2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4121625b9094e6d8a7199e2a05b0422" is_pre_election: true
I20260811 12:17:47.230022 26114 raft_consensus.cc:2454] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 356e375ace3c438ca34c15fa9997fe2a in term 0.
I20260811 12:17:47.230295 25666 raft_consensus.cc:2454] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 356e375ace3c438ca34c15fa9997fe2a in term 0.
I20260811 12:17:47.230288 26244 raft_consensus.cc:482] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:17:47.230906 25801 leader_election.cc:302] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 356e375ace3c438ca34c15fa9997fe2a, fde09377f0ce4ccf8634080cfa49d7a6; no voters:
I20260811 12:17:47.230604 26244 raft_consensus.cc:504] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.231510 26245 raft_consensus.cc:2781] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:17:47.231743 26245 raft_consensus.cc:482] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:17:47.231899 26245 raft_consensus.cc:3037] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:17:47.232918 26244 leader_election.cc:288] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4121625b9094e6d8a7199e2a05b0422 (127.7.39.1:46567), 356e375ace3c438ca34c15fa9997fe2a (127.7.39.2:33177)
I20260811 12:17:47.236511 26245 raft_consensus.cc:504] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.237478 26245 leader_election.cc:288] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [CANDIDATE]: Term 1 election: Requested vote from peers e4121625b9094e6d8a7199e2a05b0422 (127.7.39.1:46567), fde09377f0ce4ccf8634080cfa49d7a6 (127.7.39.3:44575)
I20260811 12:17:47.238500 26114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe3849ede1542b2a504ba638a87f0a5" candidate_uuid: "356e375ace3c438ca34c15fa9997fe2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fde09377f0ce4ccf8634080cfa49d7a6"
I20260811 12:17:47.238979 26114 raft_consensus.cc:3037] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:17:47.239120 25666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe3849ede1542b2a504ba638a87f0a5" candidate_uuid: "356e375ace3c438ca34c15fa9997fe2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4121625b9094e6d8a7199e2a05b0422"
I20260811 12:17:47.239368 25666 raft_consensus.cc:3037] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:17:47.243196 25666 raft_consensus.cc:2454] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 356e375ace3c438ca34c15fa9997fe2a in term 1.
I20260811 12:17:47.243724 26114 raft_consensus.cc:2454] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 356e375ace3c438ca34c15fa9997fe2a in term 1.
I20260811 12:17:47.243968 25801 leader_election.cc:302] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 356e375ace3c438ca34c15fa9997fe2a, e4121625b9094e6d8a7199e2a05b0422; no voters:
I20260811 12:17:47.244549 26245 raft_consensus.cc:2781] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:17:47.247181 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.234507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:59871 (local address 127.7.39.2:33177)
0811 12:17:47.235704 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:47.235708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:47.235724 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:17:47.235774 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:47.235778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:47.235834 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:47.235910 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:17:47.237248 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.238349 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.244141 (+ 5792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.245886 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.245988 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.246080 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:47.246701 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 12:17:47.246833 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:17:47.247406 26251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.233111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33177 (local address 127.7.39.3:59871)
0811 12:17:47.235416 (+ 2305us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:47.235433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:47.235542 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:47.235964 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:47.235968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:47.235995 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:17:47.236981 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:47.236997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.238538 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.238543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:47.243802 (+ 5259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:47.243826 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.246360 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.246367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:47.246524 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.246549 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:17:47.246632 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:17:47.246784 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2195,"mutex_wait_us":1,"thread_start_us":1852,"threads_started":1}
I20260811 12:17:47.248703 25909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe3849ede1542b2a504ba638a87f0a5" candidate_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "356e375ace3c438ca34c15fa9997fe2a" is_pre_election: true
I20260811 12:17:47.249550 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.233005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:46567 (local address 127.7.39.3:43435)
0811 12:17:47.233542 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 12:17:47.233559 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:17:47.233654 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:17:47.234112 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:17:47.234117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:17:47.234188 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:17:47.234408 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:47.234427 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.236215 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.236221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:47.244043 (+ 7822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:17:47.244061 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.247483 (+ 3422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.247491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:17:47.247733 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.247774 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:17:47.247846 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:17:47.248009 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":22}
I20260811 12:17:47.250403 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.233418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:43435 (local address 127.7.39.1:46567)
0811 12:17:47.233862 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:47.233866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:47.233879 (+ 13us) server_negotiation.cc:407] Connection header received
0811 12:17:47.233929 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:47.233932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:47.233988 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:47.234062 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:17:47.234604 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.235998 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.245310 (+ 9312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.247259 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.249822 (+ 2563us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.249981 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:47.250037 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 12:17:47.250179 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343}
I20260811 12:17:47.251130 25666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe3849ede1542b2a504ba638a87f0a5" candidate_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4121625b9094e6d8a7199e2a05b0422" is_pre_election: true
I20260811 12:17:47.251372 26245 raft_consensus.cc:686] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 1 LEADER]: Becoming Leader. State: Replica: 356e375ace3c438ca34c15fa9997fe2a, State: Running, Role: LEADER
I20260811 12:17:47.251605 25666 raft_consensus.cc:2379] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fde09377f0ce4ccf8634080cfa49d7a6 in current term 1: Already voted for candidate 356e375ace3c438ca34c15fa9997fe2a in this term.
I20260811 12:17:47.252127 26245 consensus_queue.cc:237] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } }
I20260811 12:17:47.257828 26013 leader_election.cc:302] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fde09377f0ce4ccf8634080cfa49d7a6; no voters: 356e375ace3c438ca34c15fa9997fe2a, e4121625b9094e6d8a7199e2a05b0422
I20260811 12:17:47.258450 26244 raft_consensus.cc:2726] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 12:17:47.258800 25390 catalog_manager.cc:5495] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 356e375ace3c438ca34c15fa9997fe2a (127.7.39.2). New cstate: current_term: 1 leader_uuid: "356e375ace3c438ca34c15fa9997fe2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "356e375ace3c438ca34c15fa9997fe2a" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33177 } health_report { overall_health: HEALTHY } } }
I20260811 12:17:47.334254 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.326909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45050 (local address 127.7.39.2:33177)
0811 12:17:47.327113 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:47.327118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:47.327137 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:17:47.327402 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:47.327409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:47.327668 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:47.327819 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:17:47.328362 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.329433 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.331467 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.332127 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.332232 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.332827 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:17:47.333527 (+ 700us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:17:47.333665 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:47.333818 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 12:17:47.333897 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 12:17:47.345080 26114 raft_consensus.cc:1261] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 1 FOLLOWER]: Refusing update from remote peer 356e375ace3c438ca34c15fa9997fe2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:17:47.345072 25666 raft_consensus.cc:1261] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 1 FOLLOWER]: Refusing update from remote peer 356e375ace3c438ca34c15fa9997fe2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:17:47.346351 26252 consensus_queue.cc:1035] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4121625b9094e6d8a7199e2a05b0422" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 12:17:47.347018 26245 consensus_queue.cc:1035] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fde09377f0ce4ccf8634080cfa49d7a6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:17:47.382802 25390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:17:47.383136 25390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:17:47.387424 25390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44eabbde112d471c8a69c3a726aea20b: Sending DeleteTablet for 3 replicas of tablet 4fe3849ede1542b2a504ba638a87f0a5
I20260811 12:17:47.389199 26084 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe3849ede1542b2a504ba638a87f0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:17:47 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:59908
I20260811 12:17:47.389447 25634 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe3849ede1542b2a504ba638a87f0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:17:47 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:34222
I20260811 12:17:47.390184 25875 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe3849ede1542b2a504ba638a87f0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:17:47 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:45034
I20260811 12:17:47.391178 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 25514
I20260811 12:17:47.401079 26274 tablet_replica.cc:330] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: stopping tablet replica
I20260811 12:17:47.401940 26276 tablet_replica.cc:330] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: stopping tablet replica
I20260811 12:17:47.402711 26276 raft_consensus.cc:2229] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:17:47.403376 26276 raft_consensus.cc:2258] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:17:47.403946 26274 raft_consensus.cc:2229] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:17:47.405025 26274 raft_consensus.cc:2258] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:17:47.407841 26276 ts_tablet_manager.cc:1905] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:17:47.410151 26274 ts_tablet_manager.cc:1905] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:17:47.410336 26277 tablet_replica.cc:330] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: stopping tablet replica
I20260811 12:17:47.410873 26277 raft_consensus.cc:2229] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:17:47.412640 26277 raft_consensus.cc:2258] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:17:47.416399 26276 ts_tablet_manager.cc:1918] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:17:47.416808 26276 log.cc:1196] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/4fe3849ede1542b2a504ba638a87f0a5
I20260811 12:17:47.417826 26276 ts_tablet_manager.cc:1939] T 4fe3849ede1542b2a504ba638a87f0a5 P e4121625b9094e6d8a7199e2a05b0422: Deleting consensus metadata
I20260811 12:17:47.418936 26274 ts_tablet_manager.cc:1918] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:17:47.419274 26274 log.cc:1196] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/4fe3849ede1542b2a504ba638a87f0a5
Aug 11 12:17:47 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:17:47.419617 26277 ts_tablet_manager.cc:1905] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:17:47.420047 26274 ts_tablet_manager.cc:1939] T 4fe3849ede1542b2a504ba638a87f0a5 P fde09377f0ce4ccf8634080cfa49d7a6: Deleting consensus metadata
I20260811 12:17:47.420733 25360 catalog_manager.cc:4841] TS e4121625b9094e6d8a7199e2a05b0422 (127.7.39.1:46567): tablet 4fe3849ede1542b2a504ba638a87f0a5 (table test-table [id=04b7fccc5b754f03a21ecde8f81a1dfa]) successfully deleted
I20260811 12:17:47.423290 25360 catalog_manager.cc:4841] TS fde09377f0ce4ccf8634080cfa49d7a6 (127.7.39.3:44575): tablet 4fe3849ede1542b2a504ba638a87f0a5 (table test-table [id=04b7fccc5b754f03a21ecde8f81a1dfa]) successfully deleted
I20260811 12:17:47.428686 26277 ts_tablet_manager.cc:1918] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:17:47.428938 26277 log.cc:1196] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/4fe3849ede1542b2a504ba638a87f0a5
I20260811 12:17:47.429888 26277 ts_tablet_manager.cc:1939] T 4fe3849ede1542b2a504ba638a87f0a5 P 356e375ace3c438ca34c15fa9997fe2a: Deleting consensus metadata
I20260811 12:17:47.430670 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.392772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.7.39.1:46567)
0811 12:17:47.392974 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:47.392981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:47.403185 (+ 10204us) server_negotiation.cc:407] Connection header received
0811 12:17:47.403740 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:47.403746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:47.403835 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:47.403916 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:17:47.406954 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.408043 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.411069 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.411773 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.411874 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.420716 (+ 8842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:17:47.420732 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:17:47.420751 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:17:47.420798 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:17:47.425948 (+ 5150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:47.427277 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:47.427288 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:47.427291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:47.427545 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:47.428212 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:47.428218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:47.428221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:47.428926 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:17:47.429503 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:47.429861 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 12:17:47.430148 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:17:47.431542 25734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:17:47.432142 25357 catalog_manager.cc:4841] TS 356e375ace3c438ca34c15fa9997fe2a (127.7.39.2:33177): tablet 4fe3849ede1542b2a504ba638a87f0a5 (table test-table [id=04b7fccc5b754f03a21ecde8f81a1dfa]) successfully deleted
I20260811 12:17:47.526264 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 25514
I20260811 12:17:47.549371 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 25749
Aug 11 12:17:47 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:17:47.569569 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.550203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.7.39.2:33177)
0811 12:17:47.550410 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:47.550417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:47.550439 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:17:47.550568 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:47.550574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:47.550736 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:47.550847 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:17:47.552239 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.553603 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.555022 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.555716 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.555867 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.560950 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:17:47.560972 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:17:47.560996 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:17:47.561051 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:17:47.566155 (+ 5104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:47.567121 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:47.567133 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:47.567137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:47.567352 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:47.567725 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:47.567730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:47.567734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:47.568326 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:17:47.568684 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:47.568898 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 12:17:47.569194 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:17:47.570434 25970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:17:47.684834 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 25749
I20260811 12:17:47.710708 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 25984
Aug 11 12:17:47 dist-test-slave-rmhg krb5kdc[24596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:17:47.726102 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:17:47.711499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59924 (local address 127.7.39.3:44575)
0811 12:17:47.711692 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:17:47.711697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:17:47.711717 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:17:47.711873 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:17:47.711878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:17:47.711957 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:17:47.712044 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:17:47.713433 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.714204 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.715540 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:17:47.715963 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:17:47.716062 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:17:47.720461 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:17:47.720478 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:17:47.720497 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:17:47.720536 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:17:47.723370 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:47.724046 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:47.724054 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:47.724056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:47.724196 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:17:47.724575 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:17:47.724578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:17:47.724580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:17:47.724967 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:17:47.725250 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 12:17:47.725607 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 12:17:47.725809 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:17:47.726724 26188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:17:47.820077 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 25984
I20260811 12:17:47.844282 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 25324
I20260811 12:17:47.845566 25436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:17:47.961000 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 25324
I20260811 12:17:47.995302 7324 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 12:17:48.343299 7324 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 12:17:48.343658 7324 mini_ranger.cc:66] Stopping Ranger...
I20260811 12:17:48.689728 7324 mini_ranger.cc:68] Stopped Ranger
2026-08-11 12:17:48.690 UTC [24644] LOG: received smart shutdown request
2026-08-11 12:17:48.693 UTC [24644] LOG: background worker "logical replication launcher" (PID 24657) exited with exit code 1
2026-08-11 12:17:48.694 UTC [24650] LOG: shutting down
2026-08-11 12:17:48.854 UTC [24644] LOG: database system is shut down
2026-08-11T12:17:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53406 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:17:49 dist-test-slave-rmhg krb5kdc[26300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:17:49 dist-test-slave-rmhg krb5kdc[26300](info): set up 2 sockets
Aug 11 12:17:49 dist-test-slave-rmhg krb5kdc[26300](info): commencing operation
krb5kdc: starting...
W20260811 12:17:50.218458 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:17:50 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:17:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:17:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.39.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.39.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/rangeradmin_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/rangeradmin_127.7.39.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.39.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.39.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/rangerlookup_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/rangerlookup_127.7.39.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.39.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.39.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/HTTP_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/HTTP_127.7.39.20@KRBTEST.COM.keytab.
I20260811 12:17:50.639635 7324 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDQu5eT/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/postgres -l logfile start
2026-08-11 12:17:53.172 UTC [26351] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 12:17:53.172 UTC [26351] LOG: listening on IPv4 address "127.7.39.20", port 39833
2026-08-11 12:17:53.175 UTC [26351] LOG: listening on Unix socket "/tmp/.s.PGSQL.39833"
2026-08-11 12:17:53.199 UTC [26356] LOG: database system was shut down at 2026-08-11 12:17:51 UTC
2026-08-11 12:17:53.203 UTC [26351] LOG: database system is ready to accept connections
I20260811 12:17:54.310925 7324 mini_postgres.cc:96] Postgres bound to 39833
2026-08-11 12:17:54.335 UTC [26376] FATAL: database "slave" does not exist
127.7.39.20:39833 - accepting connections
I20260811 12:17:54.336699 7324 mini_ranger.cc:161] Starting Ranger...
I20260811 12:17:54.370365 7324 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 12:17:54.898475 7324 mini_ranger.cc:87] Created ranger Postgres database
I20260811 12:17:54.898595 7324 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-admin
2026-08-11 12:17:55,204 [I] DB FLAVOR :POSTGRES
2026-08-11 12:17:55,204 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:17:55,205 [I] Checking connection..
2026-08-11 12:17:55,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 12:17:55,486 [I] Checking connection passed.
2026-08-11 12:17:55,486 [I] --------- Verifying version history table ---------
2026-08-11 12:17:55,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:17:55,760 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:17:55,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:17:56,017 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:17:56,017 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 12:17:56,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 12:17:56.264 UTC [26468] WARNING: there is no transaction in progress
2026-08-11 12:17:56,273 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 12:17:56,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:17:56,542 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 12:17:56,542 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:17:56,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 12:17:56,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 12:17:57,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-rmhg.c.gcp-upstream.internal','N') ;"
2026-08-11 12:17:57,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:17:57,577 [I] Table x_portal_user does not exist in database ranger
2026-08-11 12:17:57,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:17:57,849 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 12:17:57,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:17:58,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:17:58,386 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 12:17:58,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 12:17:59.412 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.428 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.445 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.461 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.477 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.496 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.559 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.566 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.580 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.591 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.601 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.615 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.625 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.631 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.637 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:17:59.643 UTC [26672] WARNING: there is no transaction in progress
2026-08-11 12:18:00,308 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 12:18:00,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 12:18:00,579 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 12:18:00,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:18:00,863 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 12:18:00,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:18:01,128 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 12:18:01,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:18:01,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:18:01,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-rmhg.c.gcp-upstream.internal';"
2026-08-11 12:18:01,904 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 12:18:01,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 12:18:02,152 [I] DB_PATCHES have already been applied
I20260811 12:18:02.158941 7324 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDQu5eT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 12:18:02.159045 7324 mini_ranger.cc:193] Using host: 127.7.39.20
I20260811 12:18:02.180186 7324 mini_ranger.cc:227] Ranger admin URL: http://127.7.39.20:57887
Aug 11, 2026 12:18:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:18:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:18:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:18:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:18:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:18:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:18:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 12:18:03.440271 7324 mini_ranger.cc:160] Time spent starting Ranger: real 9.104s user 0.000s sys 0.069s
Aug 11, 2026 12:18:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:18:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 12:18:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:18:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:18:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 12:18:25.350823 7324 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.39.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.39.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/rangerkms_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.39.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/rangerkms_127.7.39.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.39.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.39.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.39.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/HTTP_127.7.39.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.39.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/HTTP_127.7.39.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 12:18:25 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450705, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 12:18:25.599251 7324 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 12:18:25.626101 7324 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 12:18:26.460886 7324 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 12:18:26.461010 7324 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms
I20260811 12:18:26.461048 7324 mini_ranger_kms.cc:227] Using postgres at 127.7.39.20:39833
2026-08-11 12:18:26,811 [I] DB FLAVOR :POSTGRES
2026-08-11 12:18:26,811 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:18:26,811 [I] Checking connection
2026-08-11 12:18:26,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 12:18:27,061 [I] connection success
2026-08-11 12:18:27,061 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 12:18:27,061 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 12:18:27,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 12:18:27,308 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 12:18:27,308 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:18:27,309 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 12:18:27,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.39.20:39833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 12:18:27,570 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 12:18:28.052763 7324 mini_ranger_kms.cc:310] Created kms service
I20260811 12:18:28.261025 7324 mini_ranger_kms.cc:326] Created kudu user
I20260811 12:18:28.310020 7324 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 12:18:28 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450705, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:28.752705 7324 mini_ranger_kms.cc:384] Added ranger policy
I20260811 12:18:28.753065 7324 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDQu5eT/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDQu5eT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 12:18:28.753188 7324 mini_ranger_kms.cc:240] Using host: 127.7.39.20
I20260811 12:18:28.777053 7324 mini_ranger_kms.cc:276] Ranger KMS PID: 26981
I20260811 12:18:28.777240 7324 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.39.20:48621
Aug 11, 2026 12:18:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:18:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:18:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:18:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:18:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:18:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 12:18:31 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450711, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.39.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:31.464047 7324 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.865s user 0.076s sys 0.321s
I20260811 12:18:31.464323 7324 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 12:18:31.464541 7324 mini_ranger_kms.cc:401] 127.7.39.20:48621/kms/v1/keys
Aug 11 12:18:31 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450711, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.39.20@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
Aug 11 12:18:31 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450711, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.39.20@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
Aug 11 12:18:34 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:18:34.342653 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40997
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:40997
--ranger_config_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:48621/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:34.704834 27066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:34.705184 27066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:34.705327 27066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:34.712602 27066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:34.712740 27066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:34.712821 27066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:34.712896 27066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:34.712963 27066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:34.723275 27066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.39.20:48621/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:40997
--ranger_config_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:34.724752 27066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:34.726153 27066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:34.738315 27072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:34.738380 27077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:34.738315 27073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:34.739113 27066 server_base.cc:1034] running on GCE node
I20260811 12:18:34.739935 27066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:34.741489 27066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:34.742710 27066 hybrid_clock.cc:648] HybridClock initialized: now 1786450714742668 us; error 34 us; skew 500 ppm
I20260811 12:18:34.743058 27066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:34 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450714, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:34.751020 27066 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:18:34.752635 27066 webserver.cc:460] Webserver started at http://127.7.39.62:38305/ using document root <none> and password file <none>
I20260811 12:18:34.753383 27066 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:34.753502 27066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:34.753815 27066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:18:34 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450714, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:34.822978 27066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "064b7dd404704a8a8447abe9c2f79d93"
format_stamp: "Formatted at 2026-08-11 12:18:34 on dist-test-slave-rmhg"
server_key: "1c64920886059c0960e9808283a60ad7"
server_key_iv: "924102355f774c996b4d6d2ce2217939"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:34.823890 27066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "064b7dd404704a8a8447abe9c2f79d93"
format_stamp: "Formatted at 2026-08-11 12:18:34 on dist-test-slave-rmhg"
server_key: "1c64920886059c0960e9808283a60ad7"
server_key_iv: "924102355f774c996b4d6d2ce2217939"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:34.829924 27066 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 12:18:34.848381 27087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:34.850003 27066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:18:34.850193 27066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "064b7dd404704a8a8447abe9c2f79d93"
format_stamp: "Formatted at 2026-08-11 12:18:34 on dist-test-slave-rmhg"
server_key: "1c64920886059c0960e9808283a60ad7"
server_key_iv: "924102355f774c996b4d6d2ce2217939"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:34.850419 27066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:34.866628 27066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:34.870843 27066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:34.871160 27066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:34.890357 27066 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:40997
I20260811 12:18:34.890383 27176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:40997 every 8 connection(s)
I20260811 12:18:34.891800 27066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:18:34.895802 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:34.897138 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 27066
I20260811 12:18:34.897387 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:18:34.898044 7324 external_mini_cluster.cc:1428] Setting key 364eb822ac2fb6234ac3aaa8a98c20fd
I20260811 12:18:34.908814 27177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Bootstrap starting.
I20260811 12:18:34.912951 27177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Neither blocks nor log segments found. Creating new log.
Aug 11 12:18:34 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:34.914260 27177 log.cc:824] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:34.918099 27177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: No bootstrap required, opened a new log
I20260811 12:18:34.923364 27182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:34.900152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58084 (local address 127.7.39.62:40997)
0811 12:18:34.901035 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:34.901051 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:34.901113 (+ 62us) server_negotiation.cc:407] Connection header received
0811 12:18:34.902894 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:34.902922 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:34.903346 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:34.903675 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:34.905521 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:34.907360 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:34.909263 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:34.909997 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:34.910155 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:34.914792 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:34.914811 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:34.914838 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:34.914898 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:34.918544 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:34.919317 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:34.919327 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:34.919337 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:34.919568 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:34.919924 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:34.919929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:34.919936 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:34.920707 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:34.921035 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:34.921411 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 12:18:34.921903 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":442,"threads_started":1}
I20260811 12:18:34.927551 27177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } }
I20260811 12:18:34.927922 27177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:34.928023 27177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064b7dd404704a8a8447abe9c2f79d93, State: Initialized, Role: FOLLOWER
I20260811 12:18:34.928578 27177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } }
I20260811 12:18:34.928817 27177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:18:34.928952 27177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:18:34.929121 27177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:34.931411 27177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } }
I20260811 12:18:34.931968 27177 leader_election.cc:302] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 064b7dd404704a8a8447abe9c2f79d93; no voters:
I20260811 12:18:34.932576 27177 leader_election.cc:288] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:18:34.932740 27186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:18:34.933513 27186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 1 LEADER]: Becoming Leader. State: Replica: 064b7dd404704a8a8447abe9c2f79d93, State: Running, Role: LEADER
I20260811 12:18:34.934029 27186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } }
I20260811 12:18:34.934695 27177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:34.936931 27189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 064b7dd404704a8a8447abe9c2f79d93. Latest consensus state: current_term: 1 leader_uuid: "064b7dd404704a8a8447abe9c2f79d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } } }
I20260811 12:18:34.936901 27187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "064b7dd404704a8a8447abe9c2f79d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } } }
I20260811 12:18:34.937353 27189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:34.937412 27187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:34.939463 27196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:18:34.944991 27196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:18:34.945919 27177 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:18:34.964392 27196 catalog_manager.cc:1347] Generated new cluster ID: 2952f535a6d54c8395d36ce7ecac8496
I20260811 12:18:34.964565 27196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:18:35.015728 27196 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:18:35.017748 27196 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:18:35.033881 27196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Generated new TSK 0
I20260811 12:18:35.035147 27196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:18:35.761216 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:48621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40997
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:18:36 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450716, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 12:18:36.186919 27245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:36.187244 27245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:36.187419 27245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:36.195484 27245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:36.195626 27245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:36.195785 27245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:18:36.206440 27245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.39.20:48621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:36.207858 27245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:36.209312 27245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:36.226926 27265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:36.227633 27268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:36.229118 27245 server_base.cc:1034] running on GCE node
W20260811 12:18:36.229434 27266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:36.230355 27245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:36.231817 27245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:36.233086 27245 hybrid_clock.cc:648] HybridClock initialized: now 1786450716233014 us; error 69 us; skew 500 ppm
I20260811 12:18:36.233533 27245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:36 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450716, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:36.238292 27245 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:18:36.251600 27245 webserver.cc:460] Webserver started at http://127.7.39.1:43895/ using document root <none> and password file <none>
I20260811 12:18:36.252965 27245 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:36.253216 27245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:36.253650 27245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:18:36 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450716, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:36.277324 27245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "5441f2d65aeb4a388fa2bc64546a6940"
format_stamp: "Formatted at 2026-08-11 12:18:36 on dist-test-slave-rmhg"
server_key: "2a4ddcba28eecddef4995f7b7057cb94"
server_key_iv: "e07d6c1ab7313e1c38faed36340cbc35"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:36.278565 27245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "5441f2d65aeb4a388fa2bc64546a6940"
format_stamp: "Formatted at 2026-08-11 12:18:36 on dist-test-slave-rmhg"
server_key: "2a4ddcba28eecddef4995f7b7057cb94"
server_key_iv: "e07d6c1ab7313e1c38faed36340cbc35"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:36.286988 27245 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s
I20260811 12:18:36.345875 27288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:36.348141 27245 fs_manager.cc:725] Time spent opening block manager: real 0.026s user 0.006s sys 0.015s
I20260811 12:18:36.348361 27245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "5441f2d65aeb4a388fa2bc64546a6940"
format_stamp: "Formatted at 2026-08-11 12:18:36 on dist-test-slave-rmhg"
server_key: "2a4ddcba28eecddef4995f7b7057cb94"
server_key_iv: "e07d6c1ab7313e1c38faed36340cbc35"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:36.348628 27245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:36.398908 27245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:36.409642 27245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:36.410027 27245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:36.411387 27245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:36.413416 27245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:36.413523 27245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:36.413666 27245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:36.413724 27245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:36.498754 27245 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:40215
I20260811 12:18:36.498809 27485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:40215 every 8 connection(s)
I20260811 12:18:36.500480 27245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:18:36.509460 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 27245
I20260811 12:18:36.509693 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:18:36.510567 7324 external_mini_cluster.cc:1428] Setting key 0067f69002c4e7f4deb375515a7de1be
Aug 11 12:18:36 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450716, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:36.538542 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:36.504643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40997 (local address 127.7.39.1:46247)
0811 12:18:36.508874 (+ 4231us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:36.508929 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:36.511430 (+ 2501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:36.512876 (+ 1446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:36.512889 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:36.513604 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:36.514476 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:36.514504 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:36.516196 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:36.516201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:36.518305 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:36.518332 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:36.519614 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:36.519622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:36.519923 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:36.520749 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:36.520781 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:36.525854 (+ 5073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:36.533198 (+ 7344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:36.533214 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:36.533237 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:36.533942 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:36.534541 (+ 599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:36.534547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:36.534551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:36.534842 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:36.535590 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:36.535599 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:36.536073 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 12:18:36.536380 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 12:18:36.536985 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3893,"thread_start_us":3635,"threads_started":1}
I20260811 12:18:36.540470 27489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:36.505369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:46247 (local address 127.7.39.62:40997)
0811 12:18:36.512247 (+ 6878us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:36.512253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:36.512273 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:18:36.512327 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:36.512331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:36.512420 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:36.512524 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:36.514704 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:36.516000 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:36.518506 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:36.519156 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:36.519288 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:36.528963 (+ 9675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:36.528984 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:36.528992 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:36.529055 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:36.532891 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:36.534160 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:36.534166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:36.534169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:36.534391 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:36.535019 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:36.535024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:36.535027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:36.535388 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:36.539538 (+ 4150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:36.539711 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 12:18:36.539996 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6758,"thread_start_us":6557,"threads_started":1}
I20260811 12:18:36.542197 27486 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40997
I20260811 12:18:36.542549 27486 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:36.543424 27486 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
I20260811 12:18:36.546020 27117 ts_manager.cc:194] Registered new tserver with Master: 5441f2d65aeb4a388fa2bc64546a6940 (127.7.39.1:40215)
I20260811 12:18:36.548030 27117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:46247
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Aug 11 12:18:36 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450716, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:18:36.599365 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:48621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40997
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:18:36 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450716, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
W20260811 12:18:37.051816 27494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:37.052347 27494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:37.052565 27494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:37.062698 27494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:37.062924 27494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:37.063095 27494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:18:37.075402 27494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.39.20:48621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:37.077101 27494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:37.079077 27494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:37.117780 27508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:37.122238 27511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:37.130548 27505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:37.133067 27494 server_base.cc:1034] running on GCE node
I20260811 12:18:37.134354 27494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:37.135774 27494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:37.137013 27494 hybrid_clock.cc:648] HybridClock initialized: now 1786450717136978 us; error 97 us; skew 500 ppm
I20260811 12:18:37.137460 27494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:37 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:37.142282 27494 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:18:37.144207 27494 webserver.cc:460] Webserver started at http://127.7.39.2:36347/ using document root <none> and password file <none>
I20260811 12:18:37.145008 27494 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:37.145138 27494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:37.145506 27494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:18:37 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:37.176193 27494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "697db320a7cd43a49d87143d1bd24cf0"
format_stamp: "Formatted at 2026-08-11 12:18:37 on dist-test-slave-rmhg"
server_key: "47af1357b95db75082a24ffcd7a221f7"
server_key_iv: "52bea8a28c997a33435ae2286841d71d"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:37.178689 27494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "697db320a7cd43a49d87143d1bd24cf0"
format_stamp: "Formatted at 2026-08-11 12:18:37 on dist-test-slave-rmhg"
server_key: "47af1357b95db75082a24ffcd7a221f7"
server_key_iv: "52bea8a28c997a33435ae2286841d71d"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:37.207657 27494 fs_manager.cc:691] Time spent creating directory manager: real 0.028s user 0.019s sys 0.007s
I20260811 12:18:37.230906 27536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:37.233039 27494 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.004s
I20260811 12:18:37.233278 27494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "697db320a7cd43a49d87143d1bd24cf0"
format_stamp: "Formatted at 2026-08-11 12:18:37 on dist-test-slave-rmhg"
server_key: "47af1357b95db75082a24ffcd7a221f7"
server_key_iv: "52bea8a28c997a33435ae2286841d71d"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:37.233541 27494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:37.260807 27494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:37.266582 27494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:37.266963 27494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:37.268216 27494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:37.270386 27494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:37.270522 27494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 12:18:37.270689 27494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:37.270773 27494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:37.313297 27494 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:45887
I20260811 12:18:37.313351 27729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:45887 every 8 connection(s)
I20260811 12:18:37.315182 27494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:18:37.321768 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 27494
I20260811 12:18:37.321969 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:18:37.322650 7324 external_mini_cluster.cc:1428] Setting key 6d85397d93779d7aa88865d6fd880bdd
Aug 11 12:18:37 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
I20260811 12:18:37.373188 27737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:37.319890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:46583 (local address 127.7.39.62:40997)
0811 12:18:37.345797 (+ 25907us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:37.345804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:37.348112 (+ 2308us) server_negotiation.cc:407] Connection header received
0811 12:18:37.350778 (+ 2666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:37.350784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:37.350896 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:37.350978 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:37.353477 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.354921 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.357795 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.358439 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.358556 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:37.363150 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:37.363157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:37.363160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:37.363199 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:37.368988 (+ 5789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:37.370141 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:37.370147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:37.370150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:37.370339 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:37.370930 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:37.370935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:37.370938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:37.371321 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:37.371464 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:37.372515 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0811 12:18:37.372790 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25752,"thread_start_us":25925,"threads_started":1}
I20260811 12:18:37.376026 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:37.319234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40997 (local address 127.7.39.2:46583)
0811 12:18:37.346526 (+ 27292us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:37.346602 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:37.349347 (+ 2745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:37.351565 (+ 2218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:37.351577 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:37.352376 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:37.353255 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:37.353284 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.355289 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.355294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:37.357564 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:37.357594 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.358956 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.358964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:37.359284 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:37.360134 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:37.360167 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:37.362860 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:37.369260 (+ 6400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:37.369272 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:37.369295 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:37.369965 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:37.370501 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:37.370506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:37.370510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:37.370793 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:37.371571 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:37.371580 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:37.372048 (+ 468us) client_negotiation.cc:769] Sending connection context
0811 12:18:37.373810 (+ 1762us) client_negotiation.cc:240] Negotiation successful
0811 12:18:37.374412 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26929,"thread_start_us":30383,"threads_started":1}
I20260811 12:18:37.378368 27732 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40997
I20260811 12:18:37.378731 27732 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:37.379611 27732 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
I20260811 12:18:37.381461 27117 ts_manager.cc:194] Registered new tserver with Master: 697db320a7cd43a49d87143d1bd24cf0 (127.7.39.2:45887)
I20260811 12:18:37.382503 27117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:46583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:18:37.404302 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:48621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40997
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:18:37.551944 27486 heartbeater.cc:503] Master 127.7.39.62:40997 was elected leader, sending a full tablet report...
W20260811 12:18:37.765383 27740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:37.765722 27740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:37.765894 27740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:37.774014 27740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:37.774161 27740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:37.774320 27740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:18:37.784739 27740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.39.20:48621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:37.786172 27740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:37.787513 27740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:37.805722 27748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:37.810747 27749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:37.811311 27754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:37.812107 27740 server_base.cc:1034] running on GCE node
I20260811 12:18:37.812852 27740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:37.814013 27740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:37.815295 27740 hybrid_clock.cc:648] HybridClock initialized: now 1786450717815231 us; error 56 us; skew 500 ppm
I20260811 12:18:37.815650 27740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:37 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:37.819069 27740 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:18:37.820879 27740 webserver.cc:460] Webserver started at http://127.7.39.3:38461/ using document root <none> and password file <none>
I20260811 12:18:37.821700 27740 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:37.821831 27740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:37.822155 27740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:18:37 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:37.843508 27740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "86dfebf5684d4fafab01ea5aa424aa6b"
format_stamp: "Formatted at 2026-08-11 12:18:37 on dist-test-slave-rmhg"
server_key: "b9985c5e59314bb7e1902a6d87d3ed10"
server_key_iv: "e759b6613f483c3822c9d3d159ccb498"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:37.844520 27740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "86dfebf5684d4fafab01ea5aa424aa6b"
format_stamp: "Formatted at 2026-08-11 12:18:37 on dist-test-slave-rmhg"
server_key: "b9985c5e59314bb7e1902a6d87d3ed10"
server_key_iv: "e759b6613f483c3822c9d3d159ccb498"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:37.850875 27740 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 12:18:37.866429 27767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:37.867967 27740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:18:37.868149 27740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "86dfebf5684d4fafab01ea5aa424aa6b"
format_stamp: "Formatted at 2026-08-11 12:18:37 on dist-test-slave-rmhg"
server_key: "b9985c5e59314bb7e1902a6d87d3ed10"
server_key_iv: "e759b6613f483c3822c9d3d159ccb498"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:37.868374 27740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:37.887158 27740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:37.891364 27740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:37.891692 27740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:37.892745 27740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:37.894304 27740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:37.894387 27740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:37.894516 27740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:37.894569 27740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:37.924418 27740 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:34301
I20260811 12:18:37.924477 27976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:34301 every 8 connection(s)
I20260811 12:18:37.925827 27740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:18:37.935285 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 27740
I20260811 12:18:37.935451 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:18:37.935981 7324 external_mini_cluster.cc:1428] Setting key 93b27674731b619dcbba0047adf9c73a
Aug 11 12:18:37 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:37.946413 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:37.928519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:57903 (local address 127.7.39.62:40997)
0811 12:18:37.929068 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:37.929074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:37.930101 (+ 1027us) server_negotiation.cc:407] Connection header received
0811 12:18:37.931927 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:37.931932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:37.932003 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:37.932103 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:37.933671 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.934407 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.936120 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.936709 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.936836 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:37.940436 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:37.940442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:37.940445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:37.940518 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:37.943166 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:37.943931 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:37.943937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:37.943939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:37.944076 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:37.944556 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:37.944560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:37.944562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:37.944855 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:37.945002 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:37.945795 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 12:18:37.946057 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":216,"threads_started":1}
I20260811 12:18:37.947388 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:37.929089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40997 (local address 127.7.39.3:57903)
0811 12:18:37.929941 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:37.929981 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:37.931651 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:37.932324 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:37.932333 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:37.932780 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:37.933457 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:37.933478 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.934535 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.934539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:37.935903 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:37.935917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.936966 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.936975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:37.937250 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:37.937925 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:37.937959 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:37.940185 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:37.943320 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:37.943331 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:37.943349 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:37.943799 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:37.944190 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:37.944195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:37.944199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:37.944436 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:37.944986 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:37.944995 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:37.945407 (+ 412us) client_negotiation.cc:769] Sending connection context
0811 12:18:37.945662 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 12:18:37.946233 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":351,"threads_started":1}
I20260811 12:18:37.948800 27977 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40997
I20260811 12:18:37.949098 27977 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:37.949911 27977 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
I20260811 12:18:37.951465 27115 ts_manager.cc:194] Registered new tserver with Master: 86dfebf5684d4fafab01ea5aa424aa6b (127.7.39.3:34301)
I20260811 12:18:37.952412 27115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:57903
I20260811 12:18:37.963676 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:18:37.965451 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 27245
Aug 11 12:18:37 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:18:37.981961 27985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:37.966396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.7.39.1:40215)
0811 12:18:37.966998 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:37.967005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:37.967033 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:18:37.967112 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:37.967118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:37.967337 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:37.967469 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:37.968870 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.969991 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.971476 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:37.972046 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:37.972172 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:37.976421 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:37.976437 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:37.976458 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:37.976505 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:37.979375 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:37.980078 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:37.980086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:37.980088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:37.980250 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:37.980554 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:37.980557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:37.980559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:37.980914 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:37.981186 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:37.981427 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 12:18:37.981653 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":283,"threads_started":1}
I20260811 12:18:37.983309 27477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:38.067260 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 27245
I20260811 12:18:38.095650 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 27494
Aug 11 12:18:38 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:18:38.113057 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:38.096666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50084 (local address 127.7.39.2:45887)
0811 12:18:38.097214 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:38.097223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:38.097257 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:18:38.097321 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:38.097328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:38.097555 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:38.097724 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:38.099023 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:38.100051 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:38.101607 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:38.102239 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:38.102347 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:38.106952 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:38.106969 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:38.107008 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:38.107065 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:38.110121 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:38.111040 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:38.111049 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:38.111051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:38.111222 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:38.111549 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:38.111553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:38.111555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:38.111934 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:38.112202 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:38.112500 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 12:18:38.112731 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":256,"threads_started":1}
I20260811 12:18:38.114513 27722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:38.197947 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 27494
I20260811 12:18:38.226405 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 27740
Aug 11 12:18:38 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:18:38.244400 27991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:38.227405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.7.39.3:34301)
0811 12:18:38.228238 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:38.228246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:38.228278 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:18:38.228344 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:38.228351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:38.228587 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:38.228818 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:38.230491 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:38.231542 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:38.232858 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:38.233514 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:38.233643 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:38.238309 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:38.238325 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:38.238343 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:38.238385 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:38.241424 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:38.242180 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:38.242188 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:38.242190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:38.242334 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:38.242739 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:38.242743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:38.242745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:38.243140 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:38.243488 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:38.243759 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 12:18:38.244034 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":485,"threads_started":1}
I20260811 12:18:38.245795 27968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:38.329955 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 27740
I20260811 12:18:38.349711 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 27066
I20260811 12:18:38.351107 27171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:38.464315 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 27066
I20260811 12:18:38.498996 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40997
--webserver_interface=127.7.39.62
--webserver_port=38305
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:40997
--ranger_config_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:48621/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.39.20:48621/kms with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:38.840052 27999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:38.840378 27999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:38.840509 27999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:38.847990 27999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:38.848117 27999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:38.848191 27999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:38.848273 27999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:38.848340 27999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:38.858786 27999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.39.20:48621/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:40997
--ranger_config_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:38.860195 27999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:38.861639 27999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:38.873818 28009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:38.873900 28013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:38.873845 28011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:38.874927 27999 server_base.cc:1034] running on GCE node
I20260811 12:18:38.875701 27999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:38.877198 27999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:38.878437 27999 hybrid_clock.cc:648] HybridClock initialized: now 1786450718878369 us; error 63 us; skew 500 ppm
I20260811 12:18:38.878835 27999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:38 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450718, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:38.882083 27999 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:18:38.883644 27999 webserver.cc:460] Webserver started at http://127.7.39.62:38305/ using document root <none> and password file <none>
I20260811 12:18:38.884299 27999 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:38.884419 27999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:18:38 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450718, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:38.964517 27999 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:18:38.967921 28026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:38.969545 27999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:18:38.969822 27999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "064b7dd404704a8a8447abe9c2f79d93"
format_stamp: "Formatted at 2026-08-11 12:18:34 on dist-test-slave-rmhg"
server_key: "1c64920886059c0960e9808283a60ad7"
server_key_iv: "924102355f774c996b4d6d2ce2217939"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:38.970860 27999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:39.012255 27999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:39.016721 27999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:39.017026 27999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:39.035410 27999 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:40997
I20260811 12:18:39.035424 28115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:40997 every 8 connection(s)
I20260811 12:18:39.036993 27999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:18:39.038908 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 27999
I20260811 12:18:39.039731 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:48621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:40215
--local_ip_for_outbound_sockets=127.7.39.1
--tserver_master_addrs=127.7.39.62:40997
--webserver_port=43895
--webserver_interface=127.7.39.1
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.39.20:48621/kms with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:18:39.044164 28116 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:18:39.047623 28116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Bootstrap starting.
I20260811 12:18:39.067317 28116 log.cc:824] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:39.077468 28116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:18:39.078033 28116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Bootstrap complete.
I20260811 12:18:39.087446 28116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } }
I20260811 12:18:39.088390 28116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064b7dd404704a8a8447abe9c2f79d93, State: Initialized, Role: FOLLOWER
I20260811 12:18:39.089102 28116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } }
I20260811 12:18:39.089370 28116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:18:39.089502 28116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:18:39.089689 28116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:18:39.093750 28116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } }
I20260811 12:18:39.094277 28116 leader_election.cc:302] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 064b7dd404704a8a8447abe9c2f79d93; no voters:
I20260811 12:18:39.094901 28116 leader_election.cc:288] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:18:39.095057 28124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:18:39.096271 28124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [term 2 LEADER]: Becoming Leader. State: Replica: 064b7dd404704a8a8447abe9c2f79d93, State: Running, Role: LEADER
I20260811 12:18:39.096959 28124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } }
I20260811 12:18:39.097355 28116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:39.099730 28131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 064b7dd404704a8a8447abe9c2f79d93. Latest consensus state: current_term: 2 leader_uuid: "064b7dd404704a8a8447abe9c2f79d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } } }
I20260811 12:18:39.100128 28131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:39.100785 28126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "064b7dd404704a8a8447abe9c2f79d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064b7dd404704a8a8447abe9c2f79d93" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40997 } } }
I20260811 12:18:39.101079 28126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:39.110421 28138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:18:39.117610 28138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:18:39.119201 28138 catalog_manager.cc:1259] Loaded cluster ID: 2952f535a6d54c8395d36ce7ecac8496
I20260811 12:18:39.119408 28138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:18:39.123844 28138 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:18:39.126143 28138 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Loaded TSK: 0
I20260811 12:18:39.128099 28138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:18:39.131733 28116 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 12:18:39.472779 28122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:39.473383 28122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:39.473610 28122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:39.485447 28122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:39.485623 28122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:39.485864 28122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:18:39.502008 28122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.39.20:48621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:40215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=43895
--tserver_master_addrs=127.7.39.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:39.504083 28122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:39.506119 28122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:39.527681 28197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:39.527657 28198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:39.542899 28200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:39.549340 28122 server_base.cc:1034] running on GCE node
I20260811 12:18:39.550434 28122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:39.553820 28122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:39.555226 28122 hybrid_clock.cc:648] HybridClock initialized: now 1786450719555094 us; error 129 us; skew 500 ppm
I20260811 12:18:39.555696 28122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:39 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450719, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:39.561008 28122 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:18:39.563560 28122 webserver.cc:460] Webserver started at http://127.7.39.1:43895/ using document root <none> and password file <none>
I20260811 12:18:39.564725 28122 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:39.564913 28122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:18:39 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450719, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:39.601323 28122 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 12:18:39.605423 28212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:39.607501 28122 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:18:39.607722 28122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "5441f2d65aeb4a388fa2bc64546a6940"
format_stamp: "Formatted at 2026-08-11 12:18:36 on dist-test-slave-rmhg"
server_key: "2a4ddcba28eecddef4995f7b7057cb94"
server_key_iv: "e07d6c1ab7313e1c38faed36340cbc35"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:39.609287 28122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:39.706322 28122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:39.710805 28122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:39.711095 28122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:39.712361 28122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:39.714062 28122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:39.714187 28122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:39.714303 28122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:39.714381 28122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:39.781893 28122 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:40215
I20260811 12:18:39.781980 28394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:40215 every 8 connection(s)
I20260811 12:18:39.783875 28122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:18:39.789539 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 28122
I20260811 12:18:39.790423 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:48621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:45887
--local_ip_for_outbound_sockets=127.7.39.2
--tserver_master_addrs=127.7.39.62:40997
--webserver_port=36347
--webserver_interface=127.7.39.2
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.39.20:48621/kms with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:18:39 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450719, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:39.817840 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:39.787545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40997 (local address 127.7.39.1:36805)
0811 12:18:39.788530 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:39.788588 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:39.791029 (+ 2441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:39.793754 (+ 2725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:39.793766 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:39.794369 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:39.795197 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:39.795227 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:39.797588 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:39.797594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:39.802640 (+ 5046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:39.802675 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:39.804278 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:39.804289 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:39.804639 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:39.805505 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:39.805540 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:39.808098 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:39.812210 (+ 4112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:39.812221 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:39.812240 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:39.812826 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:39.813466 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:39.813471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:39.813473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:39.813714 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:39.814614 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:39.814625 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:39.815057 (+ 432us) client_negotiation.cc:769] Sending connection context
0811 12:18:39.815463 (+ 406us) client_negotiation.cc:240] Negotiation successful
0811 12:18:39.816220 (+ 757us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":380,"threads_started":1}
I20260811 12:18:39.817863 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:39.787254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:36805 (local address 127.7.39.62:40997)
0811 12:18:39.789710 (+ 2456us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:39.789731 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:39.789790 (+ 59us) server_negotiation.cc:407] Connection header received
0811 12:18:39.791939 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:39.791961 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:39.792509 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:39.792947 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:39.795471 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:39.797364 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:39.802888 (+ 5524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:39.803991 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:39.804204 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:39.808480 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:39.808492 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:39.808533 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:39.808598 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:39.812025 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:39.813039 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:39.813050 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:39.813060 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:39.813330 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:39.813848 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:39.813854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:39.813857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:39.814435 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:39.814642 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:39.815948 (+ 1306us) server_negotiation.cc:299] Negotiation successful
0811 12:18:39.816428 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2168,"thread_start_us":13784,"threads_started":1}
I20260811 12:18:39.822454 28396 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40997
I20260811 12:18:39.822818 28396 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:39.823734 28396 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
W20260811 12:18:39.827641 28396 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:40997 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:18:39.827912 28396 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:39.828226 28396 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
I20260811 12:18:39.829691 28396 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:39.830003 28396 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
W20260811 12:18:39.831243 28396 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:18:40.276726 28405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:40.277045 28405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:40.277243 28405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:40.285006 28405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:40.285185 28405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:40.285352 28405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:18:40.296301 28405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.39.20:48621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:45887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=36347
--tserver_master_addrs=127.7.39.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:40.297808 28405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:40.299314 28405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:40.341637 28428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:40.342499 28430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:40.350255 28437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:40.350868 28405 server_base.cc:1034] running on GCE node
I20260811 12:18:40.351895 28405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:40.365518 28405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:40.368989 28405 hybrid_clock.cc:648] HybridClock initialized: now 1786450720368839 us; error 148 us; skew 500 ppm
I20260811 12:18:40.369517 28405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:40 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:40.374090 28405 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:18:40.405817 28405 webserver.cc:460] Webserver started at http://127.7.39.2:36347/ using document root <none> and password file <none>
I20260811 12:18:40.407150 28405 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:40.407449 28405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:18:40 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
Aug 11 12:18:40 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:40.454219 28405 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.005s sys 0.004s
I20260811 12:18:40.458453 28449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:40.460510 28405 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:18:40.460742 28405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "697db320a7cd43a49d87143d1bd24cf0"
format_stamp: "Formatted at 2026-08-11 12:18:37 on dist-test-slave-rmhg"
server_key: "47af1357b95db75082a24ffcd7a221f7"
server_key_iv: "52bea8a28c997a33435ae2286841d71d"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:40.462344 28405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:40.501595 28405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:40.507464 28405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:40.507838 28405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:40.509244 28405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:40.511300 28405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:40.511428 28405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:40.511591 28405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:40.511686 28405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:40.568344 28405 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:45887
I20260811 12:18:40.568392 28650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:45887 every 8 connection(s)
I20260811 12:18:40.570019 28405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:18:40.575603 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 28405
I20260811 12:18:40.576426 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.39.20:48621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:34301
--local_ip_for_outbound_sockets=127.7.39.3
--tserver_master_addrs=127.7.39.62:40997
--webserver_port=38461
--webserver_interface=127.7.39.3
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.39.20:48621/kms with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:18:40 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:40.622099 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:40.574887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:58897 (local address 127.7.39.62:40997)
0811 12:18:40.599201 (+ 24314us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:40.599209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:40.599242 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:18:40.599882 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:40.599889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:40.599998 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:40.600110 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:40.603383 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:40.604217 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:40.606807 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:40.607487 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:40.607610 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:40.612619 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:40.612627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:40.612632 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:40.612683 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:40.617637 (+ 4954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:40.618820 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:40.618827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:40.618830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:40.618998 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:40.619633 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:40.619639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:40.619642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:40.620089 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:40.620235 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:40.621434 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0811 12:18:40.621694 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24134,"thread_start_us":24355,"threads_started":1}
I20260811 12:18:40.625605 28654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:40.574074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40997 (local address 127.7.39.2:58897)
0811 12:18:40.594098 (+ 20024us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:40.594191 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:40.599448 (+ 5257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:40.600764 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:40.600778 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:40.601501 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:40.602471 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:40.602505 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:40.604497 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:40.604503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:40.606586 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:40.606614 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:40.608065 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:40.608075 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:40.608408 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:40.609285 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:40.609322 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:40.612274 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:40.617848 (+ 5574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:40.617862 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:40.617885 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:40.618644 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:40.619179 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:40.619185 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:40.619189 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:40.619482 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:40.620351 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:40.620360 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:40.620855 (+ 495us) client_negotiation.cc:769] Sending connection context
0811 12:18:40.622673 (+ 1818us) client_negotiation.cc:240] Negotiation successful
0811 12:18:40.623329 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19555,"thread_start_us":23407,"threads_started":1}
I20260811 12:18:40.633232 28651 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40997
I20260811 12:18:40.633638 28651 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:40.634711 28651 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
I20260811 12:18:40.637313 28053 ts_manager.cc:194] Registered new tserver with Master: 697db320a7cd43a49d87143d1bd24cf0 (127.7.39.2:45887)
I20260811 12:18:40.639694 28053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:58897
I20260811 12:18:40.831461 28396 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:40.832002 28396 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
I20260811 12:18:40.833410 28053 ts_manager.cc:194] Registered new tserver with Master: 5441f2d65aeb4a388fa2bc64546a6940 (127.7.39.1:40215)
I20260811 12:18:40.834667 28053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:36805
Aug 11 12:18:40 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
W20260811 12:18:41.077739 28656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:41.078176 28656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:41.078356 28656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:41.090298 28656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:41.090492 28656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:41.090706 28656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:18:41.107381 28656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.39.20:48621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:34301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=38461
--tserver_master_addrs=127.7.39.62:40997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:41.109381 28656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:41.111368 28656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:41.129418 28668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:41.131369 28669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:41.143333 28671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:41.143548 28656 server_base.cc:1034] running on GCE node
I20260811 12:18:41.144383 28656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:41.145664 28656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:41.146921 28656 hybrid_clock.cc:648] HybridClock initialized: now 1786450721146858 us; error 58 us; skew 500 ppm
I20260811 12:18:41.147300 28656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:41 dist-test-slave-rmhg krb5kdc[26300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450721, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:41.151558 28656 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:18:41.155527 28656 webserver.cc:460] Webserver started at http://127.7.39.3:38461/ using document root <none> and password file <none>
I20260811 12:18:41.156391 28656 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:41.156564 28656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:18:41 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450721, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:41.179693 28656 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:18:41.185860 28682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:41.188336 28656 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.003s
I20260811 12:18:41.188549 28656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "86dfebf5684d4fafab01ea5aa424aa6b"
format_stamp: "Formatted at 2026-08-11 12:18:37 on dist-test-slave-rmhg"
server_key: "b9985c5e59314bb7e1902a6d87d3ed10"
server_key_iv: "e759b6613f483c3822c9d3d159ccb498"
server_key_version: "kuduclusterkey@0"
I20260811 12:18:41.190356 28656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:41.228233 28656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:41.233754 28656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:41.234124 28656 kserver.cc:163] Server-wide thread pool size limit: 3276
Aug 11 12:18:41 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for HTTP/127.7.39.20@KRBTEST.COM
I20260811 12:18:41.246704 28656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:41.248909 28656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:41.249056 28656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:41.249218 28656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:41.249312 28656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:41.449033 28656 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:34301
I20260811 12:18:41.449247 28871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:34301 every 8 connection(s)
I20260811 12:18:41.450912 28656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:18:41.458871 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 28656
Aug 11 12:18:41 dist-test-slave-rmhg krb5kdc[26300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450721, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:41.487316 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.455105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:38099 (local address 127.7.39.62:40997)
0811 12:18:41.462316 (+ 7211us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.462323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.463718 (+ 1395us) server_negotiation.cc:407] Connection header received
0811 12:18:41.466015 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.466022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.466146 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.466229 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:41.468722 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.469830 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.472290 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.472976 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.473075 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.477791 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:41.477798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:41.477803 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:41.477886 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:41.481884 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:41.483113 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:41.483119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:41.483123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:41.483291 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:41.483922 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:41.483928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:41.483931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:41.484340 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:41.484512 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.486658 (+ 2146us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.486903 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7079,"thread_start_us":6945,"threads_started":1}
I20260811 12:18:41.489219 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.454487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40997 (local address 127.7.39.3:38099)
0811 12:18:41.462898 (+ 8411us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:41.462967 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:41.465614 (+ 2647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:41.466937 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:41.466951 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:41.467598 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:41.468480 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.468506 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.470047 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.470052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.472093 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.472120 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.473486 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.473492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.473807 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.474652 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:41.474684 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:41.477490 (+ 2806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:41.482111 (+ 4621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:41.482122 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:41.482144 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:41.482929 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:41.483439 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:41.483444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:41.483447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:41.483772 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:41.484614 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:41.484624 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:41.485182 (+ 558us) client_negotiation.cc:769] Sending connection context
0811 12:18:41.485566 (+ 384us) client_negotiation.cc:240] Negotiation successful
0811 12:18:41.486224 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8079,"thread_start_us":7876,"threads_started":1}
I20260811 12:18:41.491329 28872 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40997
I20260811 12:18:41.491696 28872 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:41.492615 28872 heartbeater.cc:511] Master 127.7.39.62:40997 requested a full tablet report, sending...
I20260811 12:18:41.494343 28053 ts_manager.cc:194] Registered new tserver with Master: 86dfebf5684d4fafab01ea5aa424aa6b (127.7.39.3:34301)
I20260811 12:18:41.495530 28053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:38099
I20260811 12:18:41.502852 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.462273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56474 (local address 127.7.39.62:40997)
0811 12:18:41.472250 (+ 9977us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.472255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.477136 (+ 4881us) server_negotiation.cc:407] Connection header received
0811 12:18:41.477816 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.477821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.477927 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.478033 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:41.480354 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.481767 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.483776 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.484507 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.484642 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.488325 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:41.488331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:41.488335 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:41.488377 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:41.498660 (+ 10283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:41.499681 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:41.499687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:41.499691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:41.499855 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:41.500312 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:41.500318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:41.500321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:41.500670 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:41.500958 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.502243 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.502482 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9836,"thread_start_us":9995,"threads_started":1}
I20260811 12:18:41.505771 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:18:41.525849 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.509863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.7.39.62:40997)
0811 12:18:41.510077 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.510083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.510641 (+ 558us) server_negotiation.cc:407] Connection header received
0811 12:18:41.510843 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.510848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.510925 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.511026 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:41.513200 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.514311 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.515856 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.516479 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.516617 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.519340 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:41.519348 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:41.519352 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:41.519400 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:41.522797 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:41.523569 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:41.523575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:41.523578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:41.523757 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:41.524294 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:41.524299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:41.524302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:41.524676 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:41.524918 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.525259 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.525505 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:18:41.531803 28053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:18:41.543008 28053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:18:41.570724 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.559398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:34301 (local address 127.0.0.1:56760)
0811 12:18:41.560816 (+ 1418us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:41.560860 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:41.561051 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:41.562726 (+ 1675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:41.562734 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:41.562764 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:41.563185 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.563197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.565246 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.565251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.567410 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.567428 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.569754 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.569761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.569978 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.570026 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 12:18:41.570157 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 12:18:41.570364 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1100,"thread_start_us":936,"threads_started":1}
I20260811 12:18:41.571808 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.560015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.7.39.3:34301)
0811 12:18:41.561680 (+ 1665us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.561690 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.561736 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:18:41.561948 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.561957 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.562219 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.562407 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:41.563409 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.565044 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.567628 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.569223 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.569352 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.569602 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.571297 (+ 1695us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.571482 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1473,"thread_start_us":1326,"threads_started":1}
I20260811 12:18:41.580097 28768 tablet_service.cc:1467] Processing CreateTablet for tablet d152285cfb5e44998be89af5de7b10f3 (DEFAULT_TABLE table=test-table [id=d31e762c03ae4f5eaa5a67041e6a664e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:41.582043 28768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d152285cfb5e44998be89af5de7b10f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:41.587791 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.564634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53760)
0811 12:18:41.575392 (+ 10758us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:41.575414 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:41.575674 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:41.578320 (+ 2646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:41.578327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:41.578359 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:41.578797 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.578809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.581223 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.581229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.584833 (+ 3604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.584842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.586956 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.586963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.587130 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.587163 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:18:41.587243 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:18:41.587382 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10627,"mutex_wait_us":1,"thread_start_us":10480,"threads_started":1}
I20260811 12:18:41.588625 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.559677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:45887 (local address 127.0.0.1:44666)
0811 12:18:41.570979 (+ 11302us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:41.571002 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:41.571118 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:41.580274 (+ 9156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:41.580279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:41.580309 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:41.580512 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.580523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.582719 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.582726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.584564 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.584580 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.587844 (+ 3264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.587851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.588058 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.588090 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:18:41.588171 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:18:41.588318 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11179,"mutex_wait_us":26,"thread_start_us":19383,"threads_started":1}
I20260811 12:18:41.588681 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.565334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53760 (local address 127.7.39.1:40215)
0811 12:18:41.576258 (+ 10924us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.576266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.576302 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:18:41.576419 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.576429 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.576733 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.576963 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:41.579416 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.581030 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.585003 (+ 3973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.586426 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.586569 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.586808 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.588174 (+ 1366us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.588352 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10709,"thread_start_us":12647,"threads_started":1}
I20260811 12:18:41.589913 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.560165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.7.39.2:45887)
0811 12:18:41.577656 (+ 17491us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.577665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.577699 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:18:41.577802 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.577810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.578081 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.580724 (+ 2643us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:41.581000 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.582551 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.584756 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.587639 (+ 2883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.588999 (+ 1360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.589280 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.589356 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.589532 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17305,"thread_start_us":17135,"threads_started":1}
I20260811 12:18:41.592680 28293 tablet_service.cc:1467] Processing CreateTablet for tablet d152285cfb5e44998be89af5de7b10f3 (DEFAULT_TABLE table=test-table [id=d31e762c03ae4f5eaa5a67041e6a664e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:41.593626 28543 tablet_service.cc:1467] Processing CreateTablet for tablet d152285cfb5e44998be89af5de7b10f3 (DEFAULT_TABLE table=test-table [id=d31e762c03ae4f5eaa5a67041e6a664e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:41.594235 28293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d152285cfb5e44998be89af5de7b10f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:41.595100 28543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d152285cfb5e44998be89af5de7b10f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:41.608093 28914 tablet_bootstrap.cc:492] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Bootstrap starting.
I20260811 12:18:41.612335 28914 tablet_bootstrap.cc:654] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:41.614034 28914 log.cc:824] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:41.616698 28915 tablet_bootstrap.cc:492] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: Bootstrap starting.
I20260811 12:18:41.622228 28915 tablet_bootstrap.cc:654] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:41.623289 28914 tablet_bootstrap.cc:492] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: No bootstrap required, opened a new log
I20260811 12:18:41.623634 28914 ts_tablet_manager.cc:1397] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
I20260811 12:18:41.624100 28915 log.cc:824] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:41.631860 28918 tablet_bootstrap.cc:492] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Bootstrap starting.
I20260811 12:18:41.634361 28915 tablet_bootstrap.cc:492] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: No bootstrap required, opened a new log
I20260811 12:18:41.634799 28915 ts_tablet_manager.cc:1397] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.008s
I20260811 12:18:41.635952 28918 tablet_bootstrap.cc:654] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:41.636298 28914 raft_consensus.cc:348] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.636786 28914 raft_consensus.cc:374] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:41.636886 28914 raft_consensus.cc:729] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86dfebf5684d4fafab01ea5aa424aa6b, State: Initialized, Role: FOLLOWER
I20260811 12:18:41.637410 28918 log.cc:824] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:41.637672 28914 consensus_queue.cc:260] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.641067 28918 tablet_bootstrap.cc:492] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: No bootstrap required, opened a new log
I20260811 12:18:41.641423 28918 ts_tablet_manager.cc:1397] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260811 12:18:41.646500 28915 raft_consensus.cc:348] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.647109 28915 raft_consensus.cc:374] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:41.647264 28915 raft_consensus.cc:729] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5441f2d65aeb4a388fa2bc64546a6940, State: Initialized, Role: FOLLOWER
I20260811 12:18:41.648157 28915 consensus_queue.cc:260] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.649408 28872 heartbeater.cc:503] Master 127.7.39.62:40997 was elected leader, sending a full tablet report...
I20260811 12:18:41.650072 28914 ts_tablet_manager.cc:1428] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Time spent starting tablet: real 0.026s user 0.011s sys 0.013s
I20260811 12:18:41.651774 28918 raft_consensus.cc:348] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.652220 28651 heartbeater.cc:503] Master 127.7.39.62:40997 was elected leader, sending a full tablet report...
I20260811 12:18:41.652272 28918 raft_consensus.cc:374] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:41.652375 28918 raft_consensus.cc:729] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697db320a7cd43a49d87143d1bd24cf0, State: Initialized, Role: FOLLOWER
I20260811 12:18:41.653141 28918 consensus_queue.cc:260] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.657588 28396 heartbeater.cc:503] Master 127.7.39.62:40997 was elected leader, sending a full tablet report...
I20260811 12:18:41.658360 28915 ts_tablet_manager.cc:1428] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: Time spent starting tablet: real 0.023s user 0.013s sys 0.004s
I20260811 12:18:41.661365 28918 ts_tablet_manager.cc:1428] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Time spent starting tablet: real 0.020s user 0.015s sys 0.002s
W20260811 12:18:41.712260 28875 tablet.cc:2369] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:18:41.778182 28927 raft_consensus.cc:482] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:18:41.778501 28927 raft_consensus.cc:504] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.780256 28927 leader_election.cc:288] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 697db320a7cd43a49d87143d1bd24cf0 (127.7.39.2:45887), 5441f2d65aeb4a388fa2bc64546a6940 (127.7.39.1:40215)
I20260811 12:18:41.788664 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.780585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.7.39.3:41043)
0811 12:18:41.781022 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:41.781042 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:41.781222 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:41.782025 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:41.782030 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:41.782060 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:41.782269 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.782280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.783747 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.783753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.785867 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.785886 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.787803 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.787810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.788070 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.788107 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:18:41.788212 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 12:18:41.788370 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325}
I20260811 12:18:41.789440 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.780691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:41043 (local address 127.7.39.1:40215)
0811 12:18:41.780892 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.780899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.781214 (+ 315us) server_negotiation.cc:407] Connection header received
0811 12:18:41.781402 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.781407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.781483 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.781578 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:41.782439 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.783546 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.786108 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.787450 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.787591 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.787702 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.789023 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.789202 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:18:41.790829 28326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d152285cfb5e44998be89af5de7b10f3" candidate_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5441f2d65aeb4a388fa2bc64546a6940" is_pre_election: true
W20260811 12:18:41.791191 28399 tablet.cc:2369] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:18:41.791524 28326 raft_consensus.cc:2454] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86dfebf5684d4fafab01ea5aa424aa6b in term 0.
I20260811 12:18:41.792490 28686 leader_election.cc:302] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5441f2d65aeb4a388fa2bc64546a6940, 86dfebf5684d4fafab01ea5aa424aa6b; no voters:
I20260811 12:18:41.793079 28927 raft_consensus.cc:2781] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:18:41.793248 28927 raft_consensus.cc:482] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:18:41.793345 28927 raft_consensus.cc:3037] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:41.796655 28927 raft_consensus.cc:504] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.797415 28927 leader_election.cc:288] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [CANDIDATE]: Term 1 election: Requested vote from peers 697db320a7cd43a49d87143d1bd24cf0 (127.7.39.2:45887), 5441f2d65aeb4a388fa2bc64546a6940 (127.7.39.1:40215)
I20260811 12:18:41.797976 28326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d152285cfb5e44998be89af5de7b10f3" candidate_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5441f2d65aeb4a388fa2bc64546a6940"
I20260811 12:18:41.798213 28326 raft_consensus.cc:3037] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:41.801036 28326 raft_consensus.cc:2454] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86dfebf5684d4fafab01ea5aa424aa6b in term 1.
I20260811 12:18:41.801468 28686 leader_election.cc:302] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5441f2d65aeb4a388fa2bc64546a6940, 86dfebf5684d4fafab01ea5aa424aa6b; no voters:
I20260811 12:18:41.801736 28927 raft_consensus.cc:2781] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:18:41.802228 28927 raft_consensus.cc:686] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 1 LEADER]: Becoming Leader. State: Replica: 86dfebf5684d4fafab01ea5aa424aa6b, State: Running, Role: LEADER
I20260811 12:18:41.802977 28927 consensus_queue.cc:237] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } }
I20260811 12:18:41.806063 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.780733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:45887 (local address 127.7.39.3:44695)
0811 12:18:41.795438 (+ 14705us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:41.795470 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:41.795620 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:41.796029 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:41.796034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:41.796061 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:41.796463 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.796478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.798050 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.798056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.800019 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:41.800036 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.805081 (+ 5045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.805088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:41.805338 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.805374 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:18:41.805480 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 12:18:41.805648 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14540,"thread_start_us":14328,"threads_started":1}
I20260811 12:18:41.807694 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.781141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:44695 (local address 127.7.39.2:45887)
0811 12:18:41.781931 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.781937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.795597 (+ 13660us) server_negotiation.cc:407] Connection header received
0811 12:18:41.795787 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.795793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.795894 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.795999 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:41.796652 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.797857 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.800187 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.801474 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.801599 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.801709 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.807343 (+ 5634us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.807506 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 12:18:41.808743 28580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d152285cfb5e44998be89af5de7b10f3" candidate_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697db320a7cd43a49d87143d1bd24cf0"
I20260811 12:18:41.808723 28581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d152285cfb5e44998be89af5de7b10f3" candidate_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697db320a7cd43a49d87143d1bd24cf0" is_pre_election: true
I20260811 12:18:41.809374 28581 raft_consensus.cc:2454] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86dfebf5684d4fafab01ea5aa424aa6b in term 0.
I20260811 12:18:41.809478 28053 catalog_manager.cc:5495] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 86dfebf5684d4fafab01ea5aa424aa6b (127.7.39.3). New cstate: current_term: 1 leader_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dfebf5684d4fafab01ea5aa424aa6b" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 34301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 } health_report { overall_health: UNKNOWN } } }
W20260811 12:18:41.854485 28653 tablet.cc:2369] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:18:41.877650 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.871447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.7.39.3:34301)
0811 12:18:41.871625 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.871629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.871647 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:18:41.871813 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.871817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.871897 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.872016 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:18:41.872422 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.873220 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.875351 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.875799 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.875898 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.876524 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:18:41.877076 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:18:41.877253 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.877332 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.877414 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:18:41.886273 28581 raft_consensus.cc:3037] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:41.886448 28326 raft_consensus.cc:1261] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 1 FOLLOWER]: Refusing update from remote peer 86dfebf5684d4fafab01ea5aa424aa6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:18:41.887369 28927 consensus_queue.cc:1035] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5441f2d65aeb4a388fa2bc64546a6940" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:18:41.890273 28581 raft_consensus.cc:1261] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [term 1 FOLLOWER]: Refusing update from remote peer 86dfebf5684d4fafab01ea5aa424aa6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:18:41.891215 28927 consensus_queue.cc:1035] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "697db320a7cd43a49d87143d1bd24cf0" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 45887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:18:41.905109 28935 mvcc.cc:205] Tried to move back new op lower bound from 7317302156835098624 to 7317302156509245440. Current Snapshot: MvccSnapshot[applied={T|T < 7317302156835098624}]
I20260811 12:18:41.908766 28940 mvcc.cc:205] Tried to move back new op lower bound from 7317302156835098624 to 7317302156509245440. Current Snapshot: MvccSnapshot[applied={T|T < 7317302156835098624}]
I20260811 12:18:41.930851 28052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:18:41.931243 28052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:18:41.935897 28052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 064b7dd404704a8a8447abe9c2f79d93: Sending DeleteTablet for 3 replicas of tablet d152285cfb5e44998be89af5de7b10f3
I20260811 12:18:41.937405 28768 tablet_service.cc:1514] Processing DeleteTablet for tablet d152285cfb5e44998be89af5de7b10f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:41 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:56760
I20260811 12:18:41.937625 28543 tablet_service.cc:1514] Processing DeleteTablet for tablet d152285cfb5e44998be89af5de7b10f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:41 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:44666
I20260811 12:18:41.938138 28293 tablet_service.cc:1514] Processing DeleteTablet for tablet d152285cfb5e44998be89af5de7b10f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:41 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:53760
I20260811 12:18:41.950544 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 28122
I20260811 12:18:41.957144 28948 tablet_replica.cc:330] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: stopping tablet replica
I20260811 12:18:41.957690 28948 raft_consensus.cc:2229] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:18:41.958236 28948 raft_consensus.cc:2258] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:41.959293 28947 tablet_replica.cc:330] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: stopping tablet replica
I20260811 12:18:41.959754 28947 raft_consensus.cc:2229] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:18:41.960265 28947 raft_consensus.cc:2258] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:41.962889 28947 ts_tablet_manager.cc:1905] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:41.965328 28946 tablet_replica.cc:330] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: stopping tablet replica
I20260811 12:18:41.965783 28946 raft_consensus.cc:2229] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:18:41.966514 28946 raft_consensus.cc:2258] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:41.968757 28946 ts_tablet_manager.cc:1905] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:41.971442 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:41.952990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53768 (local address 127.7.39.1:40215)
0811 12:18:41.953227 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:41.953233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:41.953256 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:18:41.953331 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:41.953337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:41.953426 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:41.953548 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:41.955668 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.957012 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.959315 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:41.959958 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:41.960093 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:41.963425 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:41.963445 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:41.963463 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:41.963512 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:41.967821 (+ 4309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:41.968798 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:41.968808 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:41.968811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:41.969071 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:41.969603 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:41.969619 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:41.969622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:41.970166 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:41.970474 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:41.970811 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 12:18:41.971070 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:18:41.972395 28382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:41.978441 28947 ts_tablet_manager.cc:1918] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:18:41.978674 28947 log.cc:1196] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/d152285cfb5e44998be89af5de7b10f3
I20260811 12:18:41.979314 28947 ts_tablet_manager.cc:1939] T d152285cfb5e44998be89af5de7b10f3 P 697db320a7cd43a49d87143d1bd24cf0: Deleting consensus metadata
I20260811 12:18:41.979337 28946 ts_tablet_manager.cc:1918] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:18:41.979516 28946 log.cc:1196] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/d152285cfb5e44998be89af5de7b10f3
I20260811 12:18:41.980028 28946 ts_tablet_manager.cc:1939] T d152285cfb5e44998be89af5de7b10f3 P 86dfebf5684d4fafab01ea5aa424aa6b: Deleting consensus metadata
I20260811 12:18:41.981501 28033 catalog_manager.cc:4841] TS 697db320a7cd43a49d87143d1bd24cf0 (127.7.39.2:45887): tablet d152285cfb5e44998be89af5de7b10f3 (table test-table [id=d31e762c03ae4f5eaa5a67041e6a664e]) successfully deleted
I20260811 12:18:41.981784 28029 catalog_manager.cc:4841] TS 86dfebf5684d4fafab01ea5aa424aa6b (127.7.39.3:34301): tablet d152285cfb5e44998be89af5de7b10f3 (table test-table [id=d31e762c03ae4f5eaa5a67041e6a664e]) successfully deleted
I20260811 12:18:42.135306 28948 ts_tablet_manager.cc:1905] T d152285cfb5e44998be89af5de7b10f3 P 5441f2d65aeb4a388fa2bc64546a6940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:42.136137 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 28122
W20260811 12:18:42.175550 28029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.39.1:40215 (error 108)
I20260811 12:18:42.176136 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 28405
I20260811 12:18:42.197093 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.177886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.7.39.2:45887)
0811 12:18:42.178087 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:42.178093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:42.178113 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:18:42.178179 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:42.178185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:42.178261 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:42.178370 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:42.180390 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:42.181558 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:42.183557 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:42.184233 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:42.184365 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:42.188080 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:42.188103 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:42.188126 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:42.188177 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:42.192296 (+ 4119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:42.193476 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:42.193487 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:42.193491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:42.193685 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:42.194183 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:42.194189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:42.194192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:42.194782 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:42.195178 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:42.196384 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0811 12:18:42.196682 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:18:42.198585 28645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:42.213649 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.188851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53784)
0811 12:18:42.213177 (+ 24326us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:42.213355 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24150,"thread_start_us":24014,"threads_started":1}
W20260811 12:18:42.214196 28029 catalog_manager.cc:4584] TS 5441f2d65aeb4a388fa2bc64546a6940 (127.7.39.1:40215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d152285cfb5e44998be89af5de7b10f3: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
I20260811 12:18:42.268193 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.267597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53790)
0811 12:18:42.267875 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:42.268005 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 12:18:42.269604 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.269065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53806)
0811 12:18:42.269299 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:42.269406 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:18:42.322432 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 28405
I20260811 12:18:42.336311 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.335697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53818)
0811 12:18:42.336009 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:42.336124 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260811 12:18:42.337724 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.337189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53822)
0811 12:18:42.337428 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:42.337540 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260811 12:18:42.365175 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 28656
I20260811 12:18:42.382308 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.366004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.7.39.3:34301)
0811 12:18:42.366251 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:42.366257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:42.366276 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:18:42.366420 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:42.366425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:42.366496 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:42.366587 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:42.367920 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:42.368721 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:42.370253 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:42.370705 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:42.370820 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:42.373366 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:42.373383 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:42.373401 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:42.373439 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:42.377564 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:42.378578 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:42.378589 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:42.378593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:42.378805 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:42.379391 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:42.379396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:42.379399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:42.379917 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:42.380244 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:42.381639 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0811 12:18:42.381887 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 12:18:42.383092 28866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:42.419106 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.418621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53838)
0811 12:18:42.418842 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:42.418933 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 12:18:42.420203 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.419770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53840)
0811 12:18:42.419988 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:42.420058 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 12:18:42.507933 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 28656
I20260811 12:18:42.534307 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 27999
I20260811 12:18:42.535596 28107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:42.650852 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:42.650283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:40215 (local address 127.0.0.1:53844)
0811 12:18:42.650577 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:42.650679 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:18:42.651145 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 27999
I20260811 12:18:42.687201 7324 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 12:18:43.033259 7324 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 12:18:43.033535 7324 mini_ranger.cc:66] Stopping Ranger...
I20260811 12:18:43.379771 7324 mini_ranger.cc:68] Stopped Ranger
2026-08-11 12:18:43.380 UTC [26351] LOG: received smart shutdown request
2026-08-11 12:18:43.383 UTC [26351] LOG: background worker "logical replication launcher" (PID 26369) exited with exit code 1
2026-08-11 12:18:43.383 UTC [26359] LOG: shutting down
2026-08-11 12:18:43.548 UTC [26351] LOG: database system is shut down
2026-08-11T12:18:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54685 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:18:43 dist-test-slave-rmhg krb5kdc[28969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:18:43 dist-test-slave-rmhg krb5kdc[28969](info): set up 2 sockets
Aug 11 12:18:43 dist-test-slave-rmhg krb5kdc[28969](info): commencing operation
krb5kdc: starting...
W20260811 12:18:44.906975 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.003s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:18:45 dist-test-slave-rmhg krb5kdc[28969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:18:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:18:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:18:45.202246 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34259
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34259,127.7.39.61:36933,127.7.39.60:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:45.568109 28990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:45.568519 28990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:45.568676 28990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:45.576220 28990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:45.576373 28990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:45.576455 28990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:45.576547 28990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:45.576613 28990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:45.587247 28990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34259,127.7.39.61:36933,127.7.39.60:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:45.588779 28990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:45.590307 28990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:45.603147 29003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:45.603266 29005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:45.603173 29001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:45.603972 28990 server_base.cc:1034] running on GCE node
I20260811 12:18:45.604941 28990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:45.606585 28990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:45.607906 28990 hybrid_clock.cc:648] HybridClock initialized: now 1786450725607815 us; error 87 us; skew 500 ppm
I20260811 12:18:45.608301 28990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:45 dist-test-slave-rmhg krb5kdc[28969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450725, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:45.615177 28990 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:18:45.616796 28990 webserver.cc:460] Webserver started at http://127.7.39.62:39429/ using document root <none> and password file <none>
I20260811 12:18:45.617604 28990 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:45.617739 28990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:45.618058 28990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:18:45.621502 28990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "0705c22f98784f1fa5dc799d4a8f7ee9"
format_stamp: "Formatted at 2026-08-11 12:18:45 on dist-test-slave-rmhg"
server_key: "7f2a7818b522d30597816031d23755ac"
server_key_iv: "e406c3911fd4affe58ecd91819f0c7d4"
server_key_version: "encryptionkey@0"
I20260811 12:18:45.622395 28990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "0705c22f98784f1fa5dc799d4a8f7ee9"
format_stamp: "Formatted at 2026-08-11 12:18:45 on dist-test-slave-rmhg"
server_key: "7f2a7818b522d30597816031d23755ac"
server_key_iv: "e406c3911fd4affe58ecd91819f0c7d4"
server_key_version: "encryptionkey@0"
I20260811 12:18:45.628098 28990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:18:45.631894 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:45.633386 28990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:18:45.633564 28990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "0705c22f98784f1fa5dc799d4a8f7ee9"
format_stamp: "Formatted at 2026-08-11 12:18:45 on dist-test-slave-rmhg"
server_key: "7f2a7818b522d30597816031d23755ac"
server_key_iv: "e406c3911fd4affe58ecd91819f0c7d4"
server_key_version: "encryptionkey@0"
I20260811 12:18:45.633766 28990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:45.654548 28990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:45.659224 28990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:45.659518 28990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:45.678215 28990 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:34259
I20260811 12:18:45.678257 29099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:34259 every 8 connection(s)
I20260811 12:18:45.679697 28990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:18:45.682507 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 28990
I20260811 12:18:45.682760 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:18:45.683607 7324 external_mini_cluster.cc:1428] Setting key 550052329f08f92fbdab4a1bf81d7f86
I20260811 12:18:45.684296 29101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:45.693708 29101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:18:45 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450725, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.39.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.61@KRBTEST.COM" created.
I20260811 12:18:45.718494 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.699147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.7.39.62:34259)
0811 12:18:45.699895 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:45.699904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:45.699977 (+ 73us) server_negotiation.cc:407] Connection header received
0811 12:18:45.701271 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:45.701278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:45.701660 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:45.701894 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:45.703088 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:45.704891 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:45.706375 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:45.707239 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:45.707451 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:45.710931 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:45.710945 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:45.710964 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:45.711006 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:45.713883 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:45.714706 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:45.714715 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:45.714719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:45.714922 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:45.715397 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:45.715400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:45.715402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:45.715977 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:45.716154 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:45.716855 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 12:18:45.717270 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":532,"thread_start_us":322,"threads_started":1}
I20260811 12:18:45.718521 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.697690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34259 (local address 127.0.0.1:36426)
0811 12:18:45.698621 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.698685 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:45.700819 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:45.701923 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:45.701929 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:45.702510 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:45.702881 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:45.702899 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:45.705052 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:45.705056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:45.706223 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:45.706238 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:45.707480 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:45.707485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:45.707629 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:45.708342 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:45.708367 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:45.710664 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:45.714053 (+ 3389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:45.714067 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:45.714083 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:45.714532 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:45.715039 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:45.715042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:45.715045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:45.715273 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:45.716142 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:45.716151 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:45.716432 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 12:18:45.716732 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 12:18:45.717432 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":145,"thread_start_us":385,"threads_started":1}
I20260811 12:18:45.721940 29101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:18:45.723150 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.722655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57304)
0811 12:18:45.722842 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.722985 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 12:18:45.723412 29022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
I20260811 12:18:45.725143 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.724732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57310)
0811 12:18:45.724884 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.724976 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:18:45.725443 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:36933: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab.
Entry for principal kudu/127.7.39.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab.
I20260811 12:18:45.748180 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.61:36933
--webserver_interface=127.7.39.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab
--principal=kudu/127.7.39.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34259,127.7.39.61:36933,127.7.39.60:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:18:45.774919 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } attempt: 1
I20260811 12:18:45.776163 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.775711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57316)
0811 12:18:45.775909 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.776008 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:18:45.777195 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.776801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57332)
0811 12:18:45.776946 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.777022 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:18:45.777434 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:36933: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
I20260811 12:18:45.845685 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } attempt: 2
I20260811 12:18:45.846890 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.846370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57340)
0811 12:18:45.846641 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.846743 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 12:18:45.847884 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.847516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57356)
0811 12:18:45.847660 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.847750 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:18:45.848088 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:36933: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
I20260811 12:18:45.939338 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } attempt: 3
I20260811 12:18:45.940573 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.940045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57362)
0811 12:18:45.940249 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.940380 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:18:45.941685 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:45.941316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57376)
0811 12:18:45.941466 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:45.941543 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 12:18:45.941990 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:36933: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
I20260811 12:18:46.085289 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } attempt: 4
I20260811 12:18:46.086591 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.086002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57388)
0811 12:18:46.086330 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.086422 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260811 12:18:46.087677 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.087318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57404)
0811 12:18:46.087440 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.087555 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 12:18:46.087872 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:36933: Network error: Client connection negotiation failed: client connection to 127.7.39.61:36933: connect: Connection refused (error 111)
W20260811 12:18:46.106388 29109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:46.106705 29109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:46.106804 29109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:46.114216 29109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:46.114352 29109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:46.114408 29109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:46.114466 29109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:46.114545 29109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:46.124882 29109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34259,127.7.39.61:36933,127.7.39.60:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab
--principal=kudu/127.7.39.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.61:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.39.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:46.126417 29109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:46.127866 29109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:46.140486 29117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:46.140395 29122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:46.140429 29118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:46.141191 29109 server_base.cc:1034] running on GCE node
I20260811 12:18:46.142047 29109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:46.143560 29109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:46.144780 29109 hybrid_clock.cc:648] HybridClock initialized: now 1786450726144724 us; error 44 us; skew 500 ppm
I20260811 12:18:46.145181 29109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:46.148440 29109 init.cc:377] Logged in from keytab as kudu/127.7.39.61@KRBTEST.COM (short username kudu)
I20260811 12:18:46.150094 29109 webserver.cc:460] Webserver started at http://127.7.39.61:38383/ using document root <none> and password file <none>
I20260811 12:18:46.150866 29109 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:46.150991 29109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:46.151307 29109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:18:46.154539 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/data/instance:
uuid: "948452788b8b45d2a5ba9b9d71596d09"
format_stamp: "Formatted at 2026-08-11 12:18:46 on dist-test-slave-rmhg"
server_key: "d6c9b88696cc1f94648dbebfbf33c783"
server_key_iv: "9bd94152d0e967767f83a7092a3ec302"
server_key_version: "encryptionkey@0"
I20260811 12:18:46.155414 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/wal/instance:
uuid: "948452788b8b45d2a5ba9b9d71596d09"
format_stamp: "Formatted at 2026-08-11 12:18:46 on dist-test-slave-rmhg"
server_key: "d6c9b88696cc1f94648dbebfbf33c783"
server_key_iv: "9bd94152d0e967767f83a7092a3ec302"
server_key_version: "encryptionkey@0"
I20260811 12:18:46.161249 29109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:18:46.165391 29131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:46.167063 29109 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:18:46.167307 29109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/wal
uuid: "948452788b8b45d2a5ba9b9d71596d09"
format_stamp: "Formatted at 2026-08-11 12:18:46 on dist-test-slave-rmhg"
server_key: "d6c9b88696cc1f94648dbebfbf33c783"
server_key_iv: "9bd94152d0e967767f83a7092a3ec302"
server_key_version: "encryptionkey@0"
I20260811 12:18:46.167526 29109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:46.186931 29109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:46.191728 29109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:46.192021 29109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:46.211149 29109 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.61:36933
I20260811 12:18:46.211171 29222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.61:36933 every 8 connection(s)
I20260811 12:18:46.212647 29109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
I20260811 12:18:46.216785 29223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:46.217511 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 29109
I20260811 12:18:46.217746 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-1/wal/instance
I20260811 12:18:46.218349 7324 external_mini_cluster.cc:1428] Setting key fce392acbce635be4ea794959519eda9
I20260811 12:18:46.226718 29223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.39.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.60@KRBTEST.COM" created.
I20260811 12:18:46.249960 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.230435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36442 (local address 127.7.39.62:34259)
0811 12:18:46.231060 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.231067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.232234 (+ 1167us) server_negotiation.cc:407] Connection header received
0811 12:18:46.234284 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.234291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.234377 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.234477 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.236217 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.237131 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.238892 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.239338 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.239461 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.243343 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.243349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.243353 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.243391 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.246356 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.247290 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.247295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.247297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.247433 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.247948 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.247952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.247954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.248299 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.248455 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.249317 (+ 862us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.249557 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":327,"threads_started":1}
I20260811 12:18:46.251137 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.231079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34259 (local address 127.0.0.1:36442)
0811 12:18:46.232056 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.232104 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:46.233974 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:46.234775 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:46.234784 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:46.235352 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:46.236008 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.236028 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.237362 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.237367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.238722 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.238742 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.239608 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.239629 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.239871 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.240593 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:46.240627 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:46.243054 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:46.246584 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.246593 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.246610 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.247147 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.247554 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.247558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.247560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.247803 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.248452 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:46.248462 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:46.248902 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 12:18:46.249203 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 12:18:46.249764 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":419,"threads_started":1}
I20260811 12:18:46.252972 29223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:18:46.269106 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.253601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57416)
0811 12:18:46.253799 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.253820 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:46.253915 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:46.255025 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:46.255030 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:46.255466 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:46.255608 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.255616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.257404 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.257408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.258671 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.258689 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.259891 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.259898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.260128 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.260744 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:46.260761 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:46.263078 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:46.266524 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.266530 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.266535 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.266926 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.267329 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.267334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.267338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.267475 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.268121 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:46.268126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:46.268274 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 12:18:46.268399 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 12:18:46.268723 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 12:18:46.269277 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.253951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.7.39.61:36933)
0811 12:18:46.254608 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.254614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.254645 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:18:46.254690 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.254696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.254880 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.255014 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.255773 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.257218 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.258834 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.259747 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.259867 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.263316 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.263332 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.263350 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.263385 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.266372 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.267073 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.267083 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.267086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.267221 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.267577 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.267582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.267584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.267973 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.268198 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.268562 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.268844 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":497,"thread_start_us":369,"threads_started":1}
I20260811 12:18:46.272035 29223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:18:46.273514 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.272900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45258)
0811 12:18:46.273092 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.273308 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 12:18:46.273870 29132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
I20260811 12:18:46.275512 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.275149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45266)
0811 12:18:46.275318 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.275400 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:18:46.275799 29223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab.
Entry for principal kudu/127.7.39.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab.
I20260811 12:18:46.285559 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.60:43121
--webserver_interface=127.7.39.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab
--principal=kudu/127.7.39.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34259,127.7.39.61:36933,127.7.39.60:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:18:46.325209 29223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 1
I20260811 12:18:46.326531 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.325929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45280)
0811 12:18:46.326188 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.326333 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 12:18:46.327688 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.327252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45286)
0811 12:18:46.327420 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.327531 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 12:18:46.327960 29223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
I20260811 12:18:46.387144 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } attempt: 5
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450725, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:18:46.396224 29223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 2
I20260811 12:18:46.397358 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.396873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45292)
0811 12:18:46.397055 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.397176 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:18:46.398574 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.398154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45302)
0811 12:18:46.398311 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.398393 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:18:46.398891 29223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
I20260811 12:18:46.402060 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.387990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57422)
0811 12:18:46.388215 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.388239 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:46.388383 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:46.388765 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:46.388770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:46.389279 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:46.389478 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.389499 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.390555 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.390559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.391445 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.391459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.392283 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.392290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.392477 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.393208 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:46.393231 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:46.395586 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:46.399649 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.399654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.399657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.400084 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.400495 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.400499 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.400502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.400643 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.401276 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:46.401280 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:46.401424 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 12:18:46.401549 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 12:18:46.401806 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 12:18:46.402237 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.388069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57422 (local address 127.7.39.61:36933)
0811 12:18:46.388268 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.388274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.388299 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:18:46.388518 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.388524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.388614 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.388710 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.389654 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.390373 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.391609 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.392122 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.392247 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.395812 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.395817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.395821 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.395860 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.399443 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.400215 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.400221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.400225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.400388 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.400745 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.400750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.400753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.401142 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.401327 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.401703 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.401948 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 12:18:46.403406 29101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:18:46.404680 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.404219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45306)
0811 12:18:46.404417 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.404526 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 12:18:46.405695 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.405363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45318)
0811 12:18:46.405506 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.405594 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 12:18:46.405887 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
I20260811 12:18:46.457242 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 1
I20260811 12:18:46.458459 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.457954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45322)
0811 12:18:46.458173 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.458271 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 12:18:46.459494 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.459144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45328)
0811 12:18:46.459298 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.459373 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 12:18:46.459709 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
I20260811 12:18:46.490139 29223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 3
I20260811 12:18:46.491443 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.490920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45342)
0811 12:18:46.491173 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.491271 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 12:18:46.492574 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.492126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45346)
0811 12:18:46.492354 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.492438 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260811 12:18:46.492810 29223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
I20260811 12:18:46.527979 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 2
I20260811 12:18:46.529471 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.528799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45350)
0811 12:18:46.529099 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.529256 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 12:18:46.530526 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.530169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45364)
0811 12:18:46.530326 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.530406 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 12:18:46.530752 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
I20260811 12:18:46.636039 29223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 4
I20260811 12:18:46.636991 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 3
I20260811 12:18:46.637462 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.636737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45380)
0811 12:18:46.637121 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.637264 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 12:18:46.638332 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.637719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45390)
0811 12:18:46.638034 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.638149 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260811 12:18:46.639021 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.638241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45396)
0811 12:18:46.638779 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.638868 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"threads_started":1}
W20260811 12:18:46.639564 29223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
I20260811 12:18:46.639691 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.639088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45398)
0811 12:18:46.639444 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.639556 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241}
W20260811 12:18:46.639890 29101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:43121: Network error: Client connection negotiation failed: client connection to 127.7.39.60:43121: connect: Connection refused (error 111)
W20260811 12:18:46.646555 29239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:46.646903 29239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:46.647040 29239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:46.654683 29239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:46.654825 29239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:46.654910 29239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:46.654986 29239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:46.655055 29239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:46.665335 29239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34259,127.7.39.61:36933,127.7.39.60:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab
--principal=kudu/127.7.39.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.60:43121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.39.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:46.666766 29239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:46.668107 29239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:46.680611 29251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:46.680953 29247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:46.680331 29246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:46.682140 29239 server_base.cc:1034] running on GCE node
I20260811 12:18:46.683148 29239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:46.685070 29239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:46.686360 29239 hybrid_clock.cc:648] HybridClock initialized: now 1786450726686309 us; error 46 us; skew 500 ppm
I20260811 12:18:46.686793 29239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:46.690271 29239 init.cc:377] Logged in from keytab as kudu/127.7.39.60@KRBTEST.COM (short username kudu)
I20260811 12:18:46.691864 29239 webserver.cc:460] Webserver started at http://127.7.39.60:35179/ using document root <none> and password file <none>
I20260811 12:18:46.692605 29239 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:46.692736 29239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:46.693053 29239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:18:46.696293 29239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/data/instance:
uuid: "f6e3c38755754631a373362b988ea66f"
format_stamp: "Formatted at 2026-08-11 12:18:46 on dist-test-slave-rmhg"
server_key: "0a0cddf0d041b5a84891ccfffdfddff1"
server_key_iv: "41d4cb23e971d83c48774647b245aed7"
server_key_version: "encryptionkey@0"
I20260811 12:18:46.697196 29239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/wal/instance:
uuid: "f6e3c38755754631a373362b988ea66f"
format_stamp: "Formatted at 2026-08-11 12:18:46 on dist-test-slave-rmhg"
server_key: "0a0cddf0d041b5a84891ccfffdfddff1"
server_key_iv: "41d4cb23e971d83c48774647b245aed7"
server_key_version: "encryptionkey@0"
I20260811 12:18:46.703080 29239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:18:46.707145 29263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:46.708664 29239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:18:46.708866 29239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/wal
uuid: "f6e3c38755754631a373362b988ea66f"
format_stamp: "Formatted at 2026-08-11 12:18:46 on dist-test-slave-rmhg"
server_key: "0a0cddf0d041b5a84891ccfffdfddff1"
server_key_iv: "41d4cb23e971d83c48774647b245aed7"
server_key_version: "encryptionkey@0"
I20260811 12:18:46.709106 29239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:46.722322 29239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:46.727077 29239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:46.727370 29239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:46.746630 29239 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.60:43121
I20260811 12:18:46.746675 29346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.60:43121 every 8 connection(s)
I20260811 12:18:46.748181 29239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
I20260811 12:18:46.752424 29347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:46.755193 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 29239
I20260811 12:18:46.755396 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450508543501-7324-0/minicluster-data/master-2/wal/instance
I20260811 12:18:46.756050 7324 external_mini_cluster.cc:1428] Setting key 2026f7dafa6b9f8262bbe6d5d7d7f5db
I20260811 12:18:46.760910 29347 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:46.780248 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.764005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.7.39.62:34259)
0811 12:18:46.764589 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.764595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.765077 (+ 482us) server_negotiation.cc:407] Connection header received
0811 12:18:46.765291 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.765296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.765363 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.765467 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.766973 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.767799 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.769544 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.769989 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.770099 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.773922 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.773928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.773932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.773979 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.776776 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.777717 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.777722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.777725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.777914 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.778340 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.778344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.778346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.778727 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.778860 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.779600 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.779838 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":253,"threads_started":1}
I20260811 12:18:46.781554 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.758247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.7.39.60:43121)
0811 12:18:46.759128 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.759141 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.759188 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:18:46.760932 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.760960 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.761488 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.761898 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.763770 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.765747 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.767580 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.768473 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.768616 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.773504 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.773519 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.773541 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.773662 (+ 121us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.776504 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.777510 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.777519 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.777523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.777742 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.778262 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.778267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.778269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.778889 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.779216 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.779550 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.779933 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":649,"spinlock_wait_cycles":24448,"thread_start_us":398,"threads_started":1}
I20260811 12:18:46.781587 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.763972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34259 (local address 127.0.0.1:36450)
0811 12:18:46.764916 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.764955 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:46.765137 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:46.765564 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:46.765571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:46.766420 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:46.766795 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.766809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.767977 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.767981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.769395 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.769412 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.770239 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.770248 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.770440 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.771120 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:46.771154 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:46.773654 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:46.776939 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.776950 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.776963 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.777543 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.778045 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.778049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.778051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.778223 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.778875 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:46.778884 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:46.779143 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 12:18:46.779424 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 12:18:46.780060 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":146,"thread_start_us":547,"threads_started":1}
I20260811 12:18:46.784322 29347 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:46.799443 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.785038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:36933 (local address 127.0.0.1:57424)
0811 12:18:46.785339 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.785361 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:46.785480 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:46.785862 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:46.785866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:46.786357 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:46.786574 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.786588 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.787765 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.787769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.788902 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.788919 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.789802 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.789807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.789965 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.790588 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:46.790606 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:46.792875 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:46.796600 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.796605 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.796608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.797192 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.797710 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.797714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.797717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.797869 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.798516 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:46.798519 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:46.798673 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 12:18:46.798786 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 12:18:46.799087 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260811 12:18:46.799432 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.785283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57424 (local address 127.7.39.61:36933)
0811 12:18:46.785456 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.785461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.785482 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:18:46.785627 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.785633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.785719 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.785820 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.786771 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.787596 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.789082 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.789639 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.789761 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.793125 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.793134 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.793138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.793217 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.796414 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.797336 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.797343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.797347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.797571 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.797987 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.797993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.797996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.798376 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.798572 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.798921 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.799119 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:18:46.800817 29347 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:18:46.802325 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.785174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36462 (local address 127.7.39.62:34259)
0811 12:18:46.785378 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.785384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.785499 (+ 115us) server_negotiation.cc:407] Connection header received
0811 12:18:46.785634 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.785639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.785719 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.785820 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.787366 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.788202 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.789789 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.790298 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.790412 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.795163 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.795172 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.795176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.795226 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.798736 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.799687 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.799694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.799697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.799872 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.800278 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.800282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.800290 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.800647 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.800916 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.801351 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.801988 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:18:46.815454 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.801834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45424)
0811 12:18:46.802025 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.802045 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:46.802143 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:46.802546 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:46.802551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:46.803006 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:46.803172 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.803182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.804441 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.804446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.805481 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.805495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.806372 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.806378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.806565 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.807172 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:46.807193 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:46.809381 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:46.812812 (+ 3431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.812817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.812821 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.813308 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.813707 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.813711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.813713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.813853 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.814497 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:46.814501 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:46.814649 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 12:18:46.814815 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 12:18:46.815162 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:18:46.815817 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.801897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.7.39.60:43121)
0811 12:18:46.802052 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.802058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.802095 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:18:46.802240 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.802244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.802317 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.802420 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.803317 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.804286 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.805627 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.806229 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.806322 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.809591 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.809598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.809603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.809645 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.812648 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.813413 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.813419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.813422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.813578 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.813974 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.813978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.813980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.814374 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.814537 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.814916 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.815366 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":172,"server-negotiator.queue_time_us":41}
I20260811 12:18:46.817104 29101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 4
I20260811 12:18:46.821197 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.804050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.7.39.61:36933)
0811 12:18:46.804223 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.804227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.804245 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:18:46.804442 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.804448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.804528 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.804637 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.806108 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.807028 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.808304 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.808758 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.808889 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.814166 (+ 5277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.814173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.814178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.814222 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.817447 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.818880 (+ 1433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.818887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.818891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.819068 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.819576 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.819581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.819585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.819941 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.820237 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.820571 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.820819 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:18:46.821961 29347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: Bootstrap starting.
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450725, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
I20260811 12:18:46.826809 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:18:46.827039 29347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:46.828400 29347 log.cc:824] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:46.831971 29347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: No bootstrap required, opened a new log
I20260811 12:18:46.834007 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.817791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45426)
0811 12:18:46.817988 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.818008 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:46.818136 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:46.818522 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:46.818527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:46.818970 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:46.819167 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.819181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.820529 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.820535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.821948 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.821968 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.823133 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.823139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.823309 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.823992 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:46.824015 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:46.826690 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:46.830879 (+ 4189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.830885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.830889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.831449 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.831908 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.831914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.831917 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.832140 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.832906 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:46.832911 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:46.833102 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 12:18:46.833305 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 12:18:46.833626 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:18:46.834491 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.817847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.7.39.60:43121)
0811 12:18:46.817999 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.818005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.818061 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:18:46.818258 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.818263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.818339 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.819322 (+ 983us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.819470 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.820350 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.822184 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.822769 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.822886 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.826869 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.826876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.826881 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.826922 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.830683 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.831583 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.831588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.831592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.831777 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.832250 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.832255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.832259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.832763 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.832948 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.833576 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.833849 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:18:46.840623 29347 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:46.841006 29347 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:46.841096 29101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9: Bootstrap starting.
I20260811 12:18:46.841190 29347 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e3c38755754631a373362b988ea66f, State: Initialized, Role: FOLLOWER
I20260811 12:18:46.841878 29347 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:46.842790 29365 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:46.843281 29365 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:18:46.844044 29347 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:46.845873 29101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:46.848165 29101 log.cc:824] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:46.859730 29101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9: No bootstrap required, opened a new log
I20260811 12:18:46.879308 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.850702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57446 (local address 127.7.39.61:36933)
0811 12:18:46.850899 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.850906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.858914 (+ 8008us) server_negotiation.cc:407] Connection header received
0811 12:18:46.859428 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.859435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.859520 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.859595 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.861982 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.863072 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.865446 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.866105 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.866216 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.870789 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.870798 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.870804 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.870856 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.874391 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.876655 (+ 2264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.876662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.876666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.876843 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.877378 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.877384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.877388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.877810 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.878059 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.878684 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.878953 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:18:46.879513 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.846377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.7.39.60:43121)
0811 12:18:46.846570 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.846577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.858822 (+ 12245us) server_negotiation.cc:407] Connection header received
0811 12:18:46.859002 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.859009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.859149 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.859255 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.861657 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.862915 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.865475 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.866264 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.866373 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.871018 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.871026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.871031 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.871086 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.875371 (+ 4285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.876735 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.876742 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.876746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.876933 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.877492 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.877497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.877501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.877934 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.878226 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.878838 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.879131 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:18:46.879479 29101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:46.882746 29101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:46.882989 29101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0705c22f98784f1fa5dc799d4a8f7ee9, State: Initialized, Role: FOLLOWER
I20260811 12:18:46.884151 29101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:46.885579 29387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:46.886145 29387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:18:46.887192 29101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:46.891242 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.846793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.7.39.62:34259)
0811 12:18:46.846985 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.846991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.850412 (+ 3421us) server_negotiation.cc:407] Connection header received
0811 12:18:46.851609 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.851615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.851700 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.851777 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.865381 (+ 13604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.866486 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.868787 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.869508 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.869621 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.882492 (+ 12871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.882500 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.882505 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.882553 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.886498 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.887668 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.887675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.887679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.887893 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.888423 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.888429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.888433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.888894 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.889216 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.890518 (+ 1302us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.890812 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 12:18:46.896121 29386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:18:46.898234 29386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:18:46.888978 29239 master_runner.cc:186] Error getting master registration for 127.7.39.62:34259: OK, instance_id { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" instance_seqno: 1786450725672811 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:18:46.902774 29239 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 12:18:46.912660 29032 tablet.cc:2369] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:18:46.916965 29407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:18:46.917384 29407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:18:46.913934 28990 master_runner.cc:186] Error getting master registration for 127.7.39.61:36933: OK, instance_id { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" instance_seqno: 1786450726205774 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:18:46.926781 28990 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:18:46.927050 29387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:18:46.927433 29387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:46.929445 29387 leader_election.cc:288] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948452788b8b45d2a5ba9b9d71596d09 (127.7.39.61:36933), f6e3c38755754631a373362b988ea66f (127.7.39.60:43121)
I20260811 12:18:46.931531 29303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e3c38755754631a373362b988ea66f" is_pre_election: true
I20260811 12:18:46.932246 29303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0705c22f98784f1fa5dc799d4a8f7ee9 in term 0.
I20260811 12:18:46.933192 29018 leader_election.cc:302] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0705c22f98784f1fa5dc799d4a8f7ee9, f6e3c38755754631a373362b988ea66f; no voters:
I20260811 12:18:46.933964 29387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:18:46.934152 29387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:18:46.934269 29387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:46.938185 29387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:46.938782 29223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } attempt: 5
I20260811 12:18:46.942757 29303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e3c38755754631a373362b988ea66f"
I20260811 12:18:46.943034 29303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:46.949792 29303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0705c22f98784f1fa5dc799d4a8f7ee9 in term 1.
I20260811 12:18:46.952453 29018 leader_election.cc:302] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0705c22f98784f1fa5dc799d4a8f7ee9, f6e3c38755754631a373362b988ea66f; no voters:
Aug 11 12:18:46 dist-test-slave-rmhg krb5kdc[28969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
I20260811 12:18:46.953706 29410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:18:46.954658 29387 leader_election.cc:288] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [CANDIDATE]: Term 1 election: Requested vote from peers 948452788b8b45d2a5ba9b9d71596d09 (127.7.39.61:36933), f6e3c38755754631a373362b988ea66f (127.7.39.60:43121)
I20260811 12:18:46.956178 29410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [term 1 LEADER]: Becoming Leader. State: Replica: 0705c22f98784f1fa5dc799d4a8f7ee9, State: Running, Role: LEADER
I20260811 12:18:46.956928 29410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:46.963323 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.939673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:43121 (local address 127.0.0.1:45438)
0811 12:18:46.939918 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:46.939948 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:46.940095 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:46.940777 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:46.940783 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:46.941455 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:46.941786 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.941801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.944655 (+ 2854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.944661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.947119 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:46.947140 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.948658 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.948667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:46.948865 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.949832 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:46.949885 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:46.954553 (+ 4668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:46.959355 (+ 4802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.959363 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.959367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.960006 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.960660 (+ 654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:46.960666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:46.960670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:46.960867 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:46.962208 (+ 1341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:46.962215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:46.962408 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 12:18:46.962565 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 12:18:46.962903 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 12:18:46.964591 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:46.940069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.7.39.60:43121)
0811 12:18:46.940264 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:46.940270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:46.940290 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:18:46.940357 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:46.940363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:46.940440 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:46.940546 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:46.941975 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.943219 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.947291 (+ 4072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:46.948033 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:46.948206 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:46.954846 (+ 6640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:46.954854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:46.954860 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:46.954907 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:46.959116 (+ 4209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.960230 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.960237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.960241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.960443 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:46.961070 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:46.961076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:46.961079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:46.961952 (+ 873us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:46.963694 (+ 1742us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:46.963857 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 12:18:46.964144 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:18:46.970868 29223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09: Bootstrap starting.
I20260811 12:18:46.975612 29223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09: Neither blocks nor log segments found. Creating new log.
W20260811 12:18:46.979736 29272 tablet.cc:2369] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:18:46.979533 29387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0705c22f98784f1fa5dc799d4a8f7ee9. Latest consensus state: current_term: 1 leader_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:46.979898 29387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:46.980368 29223 log.cc:824] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:46.986713 29414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:18:46.989234 29223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09: No bootstrap required, opened a new log
I20260811 12:18:46.991685 29414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:18:47.005278 29223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:47.005860 29223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:47.005996 29223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948452788b8b45d2a5ba9b9d71596d09, State: Initialized, Role: FOLLOWER
I20260811 12:18:47.011174 29223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } }
I20260811 12:18:47.019804 29303 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [term 1 FOLLOWER]: Refusing update from remote peer 0705c22f98784f1fa5dc799d4a8f7ee9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:18:47.021088 29410 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:18:47.023263 29174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:47.026881 29177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948452788b8b45d2a5ba9b9d71596d09"
I20260811 12:18:47.027283 29174 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [term 1 FOLLOWER]: Refusing update from remote peer 0705c22f98784f1fa5dc799d4a8f7ee9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:18:47.028829 29178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948452788b8b45d2a5ba9b9d71596d09" is_pre_election: true
I20260811 12:18:47.029350 29178 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0705c22f98784f1fa5dc799d4a8f7ee9 in term 1.
I20260811 12:18:47.030098 29410 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:18:47.031410 29223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:47.065652 29365 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0705c22f98784f1fa5dc799d4a8f7ee9. Latest consensus state: current_term: 1 leader_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:47.065987 29365 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:18:47.068485 29416 mvcc.cc:205] Tried to move back new op lower bound from 7317302177854849024 to 7317302177622339584. Current Snapshot: MvccSnapshot[applied={T|T < 7317302177854849024}]
I20260811 12:18:47.077303 29417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:47.077838 29417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:18:47.078269 29421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0705c22f98784f1fa5dc799d4a8f7ee9. Latest consensus state: current_term: 1 leader_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:47.078761 29421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:18:47.081754 29419 mvcc.cc:205] Tried to move back new op lower bound from 7317302177854849024 to 7317302177622339584. Current Snapshot: MvccSnapshot[applied={T|T < 7317302177854849024}]
I20260811 12:18:47.084988 29411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:47.085543 29411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:47.085841 29411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:47.086062 29411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:47.086786 29430 mvcc.cc:205] Tried to move back new op lower bound from 7317302177854849024 to 7317302177622339584. Current Snapshot: MvccSnapshot[applied={T|T < 7317302177854849024}]
I20260811 12:18:47.086725 29365 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:47.087004 29365 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:18:47.089713 29414 catalog_manager.cc:1347] Generated new cluster ID: ca267eec63774b23a833c266a5726f0b
I20260811 12:18:47.089833 29414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:18:47.100579 29421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c22f98784f1fa5dc799d4a8f7ee9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34259 } } peers { permanent_uuid: "948452788b8b45d2a5ba9b9d71596d09" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 36933 } } peers { permanent_uuid: "f6e3c38755754631a373362b988ea66f" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 43121 } } }
I20260811 12:18:47.100883 29421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:18:47.111469 29414 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:18:47.113696 29414 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:18:47.128139 29414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0705c22f98784f1fa5dc799d4a8f7ee9: Generated new TSK 0
I20260811 12:18:47.130687 29414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:18:47.148684 29447 catalog_manager.cc:1259] Loaded cluster ID: ca267eec63774b23a833c266a5726f0b
I20260811 12:18:47.148887 29447 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09: loading cluster ID for follower catalog manager: success
I20260811 12:18:47.152877 29447 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09: acquiring CA information for follower catalog manager: success
I20260811 12:18:47.159407 29447 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 948452788b8b45d2a5ba9b9d71596d09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:18:47.900365 29386 catalog_manager.cc:1259] Loaded cluster ID: ca267eec63774b23a833c266a5726f0b
I20260811 12:18:47.900475 29386 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: loading cluster ID for follower catalog manager: success
I20260811 12:18:47.903156 29386 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: acquiring CA information for follower catalog manager: success
I20260811 12:18:47.905179 29386 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6e3c38755754631a373362b988ea66f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:18:48.287360 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 28990
I20260811 12:18:48.289619 29093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:48.405798 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 28990
W20260811 12:18:48.434790 29264 connection.cc:535] server connection from 127.0.0.1:45426 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 12:18:48.435220 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 29109
I20260811 12:18:48.436419 29214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:48.545567 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 29109
I20260811 12:18:48.576260 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 29239
I20260811 12:18:48.577445 29339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:48.687062 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 29239
2026-08-11T12:18:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5036 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:18:48 dist-test-slave-rmhg krb5kdc[29460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:18:48 dist-test-slave-rmhg krb5kdc[29460](info): set up 2 sockets
Aug 11 12:18:48 dist-test-slave-rmhg krb5kdc[29460](info): commencing operation
krb5kdc: starting...
W20260811 12:18:49.946802 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:18:50 dist-test-slave-rmhg krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:18:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:18:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:18:50.244393 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34875
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:50.605679 29482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:50.606030 29482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:50.606177 29482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:50.613795 29482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:50.613937 29482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:50.614028 29482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:50.614107 29482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:50.614171 29482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:50.624547 29482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:50.626009 29482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:50.627399 29482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:50.640079 29496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:50.640079 29490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:50.640161 29488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:50.640967 29482 server_base.cc:1034] running on GCE node
I20260811 12:18:50.641870 29482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:50.643376 29482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:50.644630 29482 hybrid_clock.cc:648] HybridClock initialized: now 1786450730644563 us; error 43 us; skew 500 ppm
I20260811 12:18:50.645030 29482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:50 dist-test-slave-rmhg krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:50.652143 29482 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:18:50.655439 29482 webserver.cc:460] Webserver started at http://127.7.39.62:46047/ using document root <none> and password file <none>
I20260811 12:18:50.656199 29482 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:50.656314 29482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:50.656617 29482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:18:50.659821 29482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "d3c2c6da9c304436aa61aa4a13f18957"
format_stamp: "Formatted at 2026-08-11 12:18:50 on dist-test-slave-rmhg"
server_key: "b43c4295b2eeeeb1f48f9b544eb18401"
server_key_iv: "c724b7c4173314504f5311c587b5af33"
server_key_version: "encryptionkey@0"
I20260811 12:18:50.660650 29482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "d3c2c6da9c304436aa61aa4a13f18957"
format_stamp: "Formatted at 2026-08-11 12:18:50 on dist-test-slave-rmhg"
server_key: "b43c4295b2eeeeb1f48f9b544eb18401"
server_key_iv: "c724b7c4173314504f5311c587b5af33"
server_key_version: "encryptionkey@0"
I20260811 12:18:50.666313 29482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 12:18:50.670256 29504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:50.671800 29482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:18:50.672016 29482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "d3c2c6da9c304436aa61aa4a13f18957"
format_stamp: "Formatted at 2026-08-11 12:18:50 on dist-test-slave-rmhg"
server_key: "b43c4295b2eeeeb1f48f9b544eb18401"
server_key_iv: "c724b7c4173314504f5311c587b5af33"
server_key_version: "encryptionkey@0"
I20260811 12:18:50.672231 29482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:50.689304 29482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:50.693756 29482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:50.694056 29482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:50.712939 29482 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:34875
I20260811 12:18:50.712962 29598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:34875 every 8 connection(s)
I20260811 12:18:50.714450 29482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:18:50.715214 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 29482
I20260811 12:18:50.715427 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:18:50.716210 7324 external_mini_cluster.cc:1428] Setting key 9e1668bf98c4c49bdea5b17e649bae2b
I20260811 12:18:50.718991 29602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:50.729971 29602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957: Bootstrap starting.
Aug 11 12:18:50 dist-test-slave-rmhg krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:50.733476 29602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:50.734615 29602 log.cc:824] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:50.738300 29602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957: No bootstrap required, opened a new log
I20260811 12:18:50.739763 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:50.718145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48890 (local address 127.7.39.62:34875)
0811 12:18:50.718961 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:50.718974 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:50.719022 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:18:50.720740 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:50.720764 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:50.721209 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:50.721614 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:50.723210 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:50.724603 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:50.726025 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:50.726745 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:50.726965 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:50.731295 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:50.731303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:50.731321 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:50.731369 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:50.734518 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:50.735668 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:50.735676 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:50.735683 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:50.735899 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:50.736344 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:50.736349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:50.736353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:50.737131 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:50.737493 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:50.737874 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 12:18:50.738351 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":371,"threads_started":1}
I20260811 12:18:50.744199 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:18:50.748785 29602 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c2c6da9c304436aa61aa4a13f18957" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34875 } }
I20260811 12:18:50.749179 29602 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:50.749284 29602 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c2c6da9c304436aa61aa4a13f18957, State: Initialized, Role: FOLLOWER
I20260811 12:18:50.749809 29602 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c2c6da9c304436aa61aa4a13f18957" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34875 } }
I20260811 12:18:50.750018 29602 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:18:50.750119 29602 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:18:50.750252 29602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:50.752454 29602 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c2c6da9c304436aa61aa4a13f18957" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34875 } }
I20260811 12:18:50.752935 29602 leader_election.cc:302] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3c2c6da9c304436aa61aa4a13f18957; no voters:
I20260811 12:18:50.753541 29602 leader_election.cc:288] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:18:50.753679 29608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:18:50.754321 29608 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [term 1 LEADER]: Becoming Leader. State: Replica: d3c2c6da9c304436aa61aa4a13f18957, State: Running, Role: LEADER
I20260811 12:18:50.754837 29608 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c2c6da9c304436aa61aa4a13f18957" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34875 } }
I20260811 12:18:50.755555 29602 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:50.757678 29609 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3c2c6da9c304436aa61aa4a13f18957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c2c6da9c304436aa61aa4a13f18957" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34875 } } }
I20260811 12:18:50.758215 29609 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:50.758145 29610 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3c2c6da9c304436aa61aa4a13f18957. Latest consensus state: current_term: 1 leader_uuid: "d3c2c6da9c304436aa61aa4a13f18957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c2c6da9c304436aa61aa4a13f18957" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34875 } } }
I20260811 12:18:50.758466 29610 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:50.760310 29618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:18:50.766060 29618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:18:50.775036 29618 catalog_manager.cc:1347] Generated new cluster ID: 09f044e7cad14bd89233ca5b7fa76443
I20260811 12:18:50.775174 29618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:18:50.785938 29618 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:18:50.788064 29618 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:18:50.799397 29618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3c2c6da9c304436aa61aa4a13f18957: Generated new TSK 0
I20260811 12:18:50.800477 29618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:18:50.858250 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 29482
I20260811 12:18:50.859683 29589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:50.959285 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 29482
2026-08-11T12:18:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2271 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:18:51 dist-test-slave-rmhg krb5kdc[29642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:18:51 dist-test-slave-rmhg krb5kdc[29642](info): set up 2 sockets
Aug 11 12:18:51 dist-test-slave-rmhg krb5kdc[29642](info): commencing operation
krb5kdc: starting...
W20260811 12:18:52.202466 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:18:52 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:18:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:18:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:18:52.500411 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34191
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:52.854003 29667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:52.854302 29667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:52.854409 29667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:52.861770 29667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:52.861894 29667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:52.861943 29667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:52.861999 29667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:52.862043 29667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:52.872148 29667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:52.873528 29667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:52.874892 29667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:52.886758 29676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:52.886812 29677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:52.886780 29682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:52.888111 29667 server_base.cc:1034] running on GCE node
I20260811 12:18:52.889053 29667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:52.890738 29667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:52.891968 29667 hybrid_clock.cc:648] HybridClock initialized: now 1786450732891917 us; error 47 us; skew 500 ppm
I20260811 12:18:52.892346 29667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:52 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450732, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:52.895664 29667 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:18:52.897429 29667 webserver.cc:460] Webserver started at http://127.7.39.62:34439/ using document root <none> and password file <none>
I20260811 12:18:52.898172 29667 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:52.898293 29667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:52.898613 29667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:18:52.901932 29667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "92995140ec2b42e6af70790eb20f5953"
format_stamp: "Formatted at 2026-08-11 12:18:52 on dist-test-slave-rmhg"
server_key: "6b9ed4016c1b98dc33235447974ea044"
server_key_iv: "60b704c762d50778b2c558f65137d7a9"
server_key_version: "encryptionkey@0"
I20260811 12:18:52.902734 29667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "92995140ec2b42e6af70790eb20f5953"
format_stamp: "Formatted at 2026-08-11 12:18:52 on dist-test-slave-rmhg"
server_key: "6b9ed4016c1b98dc33235447974ea044"
server_key_iv: "60b704c762d50778b2c558f65137d7a9"
server_key_version: "encryptionkey@0"
I20260811 12:18:52.908375 29667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:18:52.912281 29692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:52.913794 29667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:18:52.913975 29667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "92995140ec2b42e6af70790eb20f5953"
format_stamp: "Formatted at 2026-08-11 12:18:52 on dist-test-slave-rmhg"
server_key: "6b9ed4016c1b98dc33235447974ea044"
server_key_iv: "60b704c762d50778b2c558f65137d7a9"
server_key_version: "encryptionkey@0"
I20260811 12:18:52.914196 29667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:52.937734 29667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:52.941984 29667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:52.942266 29667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:52.960532 29667 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:34191
I20260811 12:18:52.960575 29775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:34191 every 8 connection(s)
I20260811 12:18:52.961992 29667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:18:52.965823 29776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:52.970508 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 29667
I20260811 12:18:52.970714 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:18:52.971405 7324 external_mini_cluster.cc:1428] Setting key 41b4fe2b4631b2f619097e6dbd648a6e
I20260811 12:18:52.978585 29776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Bootstrap starting.
I20260811 12:18:52.982514 29776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:52.983743 29776 log.cc:824] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:52.986967 29776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: No bootstrap required, opened a new log
Aug 11 12:18:52 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:52.997633 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:52.973635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42818 (local address 127.7.39.62:34191)
0811 12:18:52.974523 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:52.974540 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:52.974599 (+ 59us) server_negotiation.cc:407] Connection header received
0811 12:18:52.977076 (+ 2477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:52.977121 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:52.977712 (+ 591us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:52.978215 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:52.979622 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:52.981715 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:52.983188 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:52.984255 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:52.984484 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:52.988800 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:52.988811 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:52.988832 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:52.988881 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:52.992900 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:52.993613 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:52.993623 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:52.993633 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:52.993852 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:52.994222 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:52.994228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:52.994231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:52.994996 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:52.995328 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:52.995666 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 12:18:52.996182 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":362,"threads_started":1}
I20260811 12:18:52.997371 29776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:52.997879 29776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:52.998030 29776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92995140ec2b42e6af70790eb20f5953, State: Initialized, Role: FOLLOWER
I20260811 12:18:52.998646 29776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:52.998875 29776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:18:52.999009 29776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:18:52.999183 29776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:53.001685 29776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:53.002190 29776 leader_election.cc:302] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92995140ec2b42e6af70790eb20f5953; no voters:
I20260811 12:18:53.002818 29776 leader_election.cc:288] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:18:53.002967 29786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:18:53.003649 29786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 1 LEADER]: Becoming Leader. State: Replica: 92995140ec2b42e6af70790eb20f5953, State: Running, Role: LEADER
I20260811 12:18:53.004300 29786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:53.005038 29776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:53.007385 29787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92995140ec2b42e6af70790eb20f5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } } }
I20260811 12:18:53.007476 29788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92995140ec2b42e6af70790eb20f5953. Latest consensus state: current_term: 1 leader_uuid: "92995140ec2b42e6af70790eb20f5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } } }
I20260811 12:18:53.007925 29788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:53.007952 29787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:53.009603 29799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:18:53.015673 29799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:18:53.022405 29812 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:18:53.025803 29799 catalog_manager.cc:1347] Generated new cluster ID: 860831a079694580aa8a3fab1a01286d
I20260811 12:18:53.025919 29799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:18:53.052378 29799 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:18:53.054394 29799 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:18:53.073629 29799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Generated new TSK 0
I20260811 12:18:53.074762 29799 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:18:53.151252 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:34191
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:53.504452 29818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:53.504818 29818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:53.504961 29818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:53.512882 29818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:53.513043 29818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:53.513232 29818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:18:53.523870 29818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:53.525373 29818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:53.526798 29818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:53.540614 29827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:53.540658 29829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:53.540768 29825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:53.541829 29818 server_base.cc:1034] running on GCE node
I20260811 12:18:53.542555 29818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:53.543545 29818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:53.544775 29818 hybrid_clock.cc:648] HybridClock initialized: now 1786450733544712 us; error 49 us; skew 500 ppm
I20260811 12:18:53.545097 29818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:53 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450733, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:53.548393 29818 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:18:53.549865 29818 webserver.cc:460] Webserver started at http://127.7.39.1:39535/ using document root <none> and password file <none>
I20260811 12:18:53.550545 29818 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:53.550657 29818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:53.550969 29818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:18:53.553978 29818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "3d013cbbb9ad4920aadaebba30065933"
format_stamp: "Formatted at 2026-08-11 12:18:53 on dist-test-slave-rmhg"
server_key: "c6a7388778ecc2aaba598d55d4c607f3"
server_key_iv: "026c86de5a793cb58599930e00a2c0e1"
server_key_version: "encryptionkey@0"
I20260811 12:18:53.554715 29818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "3d013cbbb9ad4920aadaebba30065933"
format_stamp: "Formatted at 2026-08-11 12:18:53 on dist-test-slave-rmhg"
server_key: "c6a7388778ecc2aaba598d55d4c607f3"
server_key_iv: "026c86de5a793cb58599930e00a2c0e1"
server_key_version: "encryptionkey@0"
I20260811 12:18:53.560120 29818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:18:53.563968 29837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:53.565632 29818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:18:53.565845 29818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "3d013cbbb9ad4920aadaebba30065933"
format_stamp: "Formatted at 2026-08-11 12:18:53 on dist-test-slave-rmhg"
server_key: "c6a7388778ecc2aaba598d55d4c607f3"
server_key_iv: "026c86de5a793cb58599930e00a2c0e1"
server_key_version: "encryptionkey@0"
I20260811 12:18:53.566141 29818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:53.584081 29818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:53.588616 29818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:53.588904 29818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:53.589967 29818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:53.591421 29818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:53.591497 29818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:53.591598 29818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:53.591655 29818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:53.622771 29818 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:39543
I20260811 12:18:53.622793 30021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:39543 every 8 connection(s)
I20260811 12:18:53.624100 29818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:18:53.632593 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 29818
I20260811 12:18:53.632807 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:18:53.633525 7324 external_mini_cluster.cc:1428] Setting key ec8d12ad52c6e8809073a77ffeec2dd9
Aug 11 12:18:53 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450733, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:53.644268 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:53.626732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:36999 (local address 127.7.39.62:34191)
0811 12:18:53.627230 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:53.627238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:53.628247 (+ 1009us) server_negotiation.cc:407] Connection header received
0811 12:18:53.630150 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:53.630155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:53.630231 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:53.630332 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:53.631889 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:53.632646 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:53.634414 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:53.634866 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:53.634982 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:53.638549 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:53.638555 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:53.638558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:53.638595 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:53.641124 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:53.641919 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:53.641923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:53.641926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:53.642047 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:53.642482 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:53.642486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:53.642488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:53.642768 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:53.642902 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:53.643656 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 12:18:53.643897 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260811 12:18:53.645339 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:53.627305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:36999)
0811 12:18:53.628051 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:53.628093 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:53.629861 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:53.630555 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:53.630564 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:53.631135 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:53.631742 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:53.631761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:53.632821 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:53.632825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:53.634242 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:53.634256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:53.635110 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:53.635118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:53.635360 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:53.636022 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:53.636050 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:53.638294 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:53.641315 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:53.641325 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:53.641343 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:53.641781 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:53.642160 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:53.642165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:53.642168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:53.642365 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:53.642893 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:53.642901 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:53.643266 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 12:18:53.643517 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 12:18:53.644076 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":286,"threads_started":1}
I20260811 12:18:53.647082 30022 heartbeater.cc:347] Connected to a master server at 127.7.39.62:34191
I20260811 12:18:53.647399 30022 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:53.648161 30022 heartbeater.cc:511] Master 127.7.39.62:34191 requested a full tablet report, sending...
I20260811 12:18:53.650308 29722 ts_manager.cc:194] Registered new tserver with Master: 3d013cbbb9ad4920aadaebba30065933 (127.7.39.1:39543)
I20260811 12:18:53.651772 29722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:36999
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:18:53.696686 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:34191
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:54.042121 30028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:54.042433 30028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:54.042527 30028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:54.049862 30028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:54.049988 30028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:54.050110 30028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:18:54.060431 30028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:54.061856 30028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:54.063208 30028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:54.076543 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:54.076556 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:54.076550 30041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:54.077405 30028 server_base.cc:1034] running on GCE node
I20260811 12:18:54.078042 30028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:54.078976 30028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:54.080221 30028 hybrid_clock.cc:648] HybridClock initialized: now 1786450734080163 us; error 61 us; skew 500 ppm
I20260811 12:18:54.080572 30028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:54 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450734, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:54.083865 30028 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:18:54.085665 30028 webserver.cc:460] Webserver started at http://127.7.39.2:35601/ using document root <none> and password file <none>
I20260811 12:18:54.086421 30028 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:54.086552 30028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:54.086874 30028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:18:54.090164 30028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "b21705b13fc94aed9b598a38da54d6ba"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "43bfbbd88f831912b87a3755985e477b"
server_key_iv: "cd733863f780e11a35d60824342ac0e6"
server_key_version: "encryptionkey@0"
I20260811 12:18:54.090989 30028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "b21705b13fc94aed9b598a38da54d6ba"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "43bfbbd88f831912b87a3755985e477b"
server_key_iv: "cd733863f780e11a35d60824342ac0e6"
server_key_version: "encryptionkey@0"
I20260811 12:18:54.096735 30028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:18:54.100656 30054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:54.102245 30028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:18:54.102423 30028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "b21705b13fc94aed9b598a38da54d6ba"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "43bfbbd88f831912b87a3755985e477b"
server_key_iv: "cd733863f780e11a35d60824342ac0e6"
server_key_version: "encryptionkey@0"
I20260811 12:18:54.102628 30028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:54.133977 30028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:54.138461 30028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:54.138775 30028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:54.139873 30028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:54.141404 30028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:54.141494 30028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:54.141613 30028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:54.141675 30028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:54.173359 30028 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:44745
I20260811 12:18:54.173401 30243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:44745 every 8 connection(s)
I20260811 12:18:54.174710 30028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:18:54.176784 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 30028
I20260811 12:18:54.176966 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:18:54.177603 7324 external_mini_cluster.cc:1428] Setting key 699591f2a5a9333892501d7fb2746d51
Aug 11 12:18:54 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450734, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:54.196465 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.178523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:56781 (local address 127.7.39.62:34191)
0811 12:18:54.179094 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:54.179100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:54.179281 (+ 181us) server_negotiation.cc:407] Connection header received
0811 12:18:54.181301 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:54.181306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:54.181388 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:54.181499 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:54.183057 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.184056 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.185727 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.186335 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.186450 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.190000 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:54.190007 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:54.190011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:54.190056 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:54.192922 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:54.193918 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:54.193922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:54.193925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:54.194062 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:54.194546 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:54.194549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:54.194551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:54.194835 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:54.194946 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:54.195964 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0811 12:18:54.196174 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":758,"threads_started":1}
I20260811 12:18:54.197932 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.178195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:56781)
0811 12:18:54.179107 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:54.179177 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:54.180995 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:54.181716 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:54.181724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:54.182274 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:54.182868 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.182888 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.184203 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.184207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.185577 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.185592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.186537 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.186542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.186756 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.187406 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:54.187429 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:54.189732 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:54.193121 (+ 3389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:54.193132 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:54.193177 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:54.193803 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:54.194172 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:54.194178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:54.194181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:54.194452 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:54.194978 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:54.194988 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:54.195492 (+ 504us) client_negotiation.cc:769] Sending connection context
0811 12:18:54.195800 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 12:18:54.196409 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":372,"threads_started":1}
I20260811 12:18:54.199651 30245 heartbeater.cc:347] Connected to a master server at 127.7.39.62:34191
I20260811 12:18:54.199985 30245 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:54.200783 30245 heartbeater.cc:511] Master 127.7.39.62:34191 requested a full tablet report, sending...
I20260811 12:18:54.202454 29722 ts_manager.cc:194] Registered new tserver with Master: b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745)
I20260811 12:18:54.203298 29722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:56781
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:18:54.245428 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:34191
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:54.596181 30253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:54.596482 30253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:54.596617 30253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:54.603852 30253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:54.603967 30253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:54.604120 30253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:18:54.614216 30253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:54.615559 30253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:54.616915 30253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:54.630796 30260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:54.630796 30263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:54.630908 30266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:54.631632 30253 server_base.cc:1034] running on GCE node
I20260811 12:18:54.632205 30253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:54.633109 30253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:54.634353 30253 hybrid_clock.cc:648] HybridClock initialized: now 1786450734634302 us; error 43 us; skew 500 ppm
I20260811 12:18:54.634696 30253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:54 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450734, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:54.637995 30253 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:18:54.639652 30253 webserver.cc:460] Webserver started at http://127.7.39.3:38963/ using document root <none> and password file <none>
I20260811 12:18:54.640441 30253 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:54.640575 30253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:54.640890 30253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:18:54.644106 30253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "64bdd35fb68746bb83913ec4f840f35d"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "fdf69fb0e8aa44886ee820bd4ef7f2d1"
server_key_iv: "11fccc9fe940e3493d570233580be3f7"
server_key_version: "encryptionkey@0"
I20260811 12:18:54.644982 30253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "64bdd35fb68746bb83913ec4f840f35d"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "fdf69fb0e8aa44886ee820bd4ef7f2d1"
server_key_iv: "11fccc9fe940e3493d570233580be3f7"
server_key_version: "encryptionkey@0"
I20260811 12:18:54.650727 30253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:18:54.654491 30280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:54.654742 30022 heartbeater.cc:503] Master 127.7.39.62:34191 was elected leader, sending a full tablet report...
I20260811 12:18:54.656167 30253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:18:54.656363 30253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "64bdd35fb68746bb83913ec4f840f35d"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "fdf69fb0e8aa44886ee820bd4ef7f2d1"
server_key_iv: "11fccc9fe940e3493d570233580be3f7"
server_key_version: "encryptionkey@0"
I20260811 12:18:54.656610 30253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:54.687276 30253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:54.691375 30253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:54.691644 30253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:54.692595 30253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:54.694155 30253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:54.694237 30253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:54.694334 30253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:54.694376 30253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:54.724399 30253 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:33901
I20260811 12:18:54.724463 30462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:33901 every 8 connection(s)
I20260811 12:18:54.725723 30253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:18:54.734664 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 30253
I20260811 12:18:54.734867 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:18:54.735436 7324 external_mini_cluster.cc:1428] Setting key d7dcb59ac2806ea244c20a9764ddd8fb
Aug 11 12:18:54 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450734, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:54.746289 30466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.728265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:55769 (local address 127.7.39.62:34191)
0811 12:18:54.728739 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:54.728745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:54.729883 (+ 1138us) server_negotiation.cc:407] Connection header received
0811 12:18:54.731734 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:54.731739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:54.731809 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:54.731911 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:54.733428 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.734208 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.736359 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.736852 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.736963 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.740483 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:54.740488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:54.740492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:54.740540 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:54.743210 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:54.744045 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:54.744048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:54.744051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:54.744178 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:54.744616 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:54.744619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:54.744621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:54.744886 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:54.745000 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:54.745806 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 12:18:54.745998 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":209,"threads_started":1}
I20260811 12:18:54.747375 30467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.728928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:55769)
0811 12:18:54.729695 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:54.729736 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:54.731450 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:54.732120 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:54.732129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:54.732651 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:54.733277 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.733299 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.734375 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.734381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.736229 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.736243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.737073 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.737081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.737354 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.738010 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:54.738034 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:54.740262 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:54.743389 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:54.743399 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:54.743415 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:54.743925 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:54.744266 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:54.744269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:54.744271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:54.744511 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:54.745004 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:54.745010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:54.745445 (+ 435us) client_negotiation.cc:769] Sending connection context
0811 12:18:54.745708 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 12:18:54.746154 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":321,"threads_started":1}
I20260811 12:18:54.748972 30463 heartbeater.cc:347] Connected to a master server at 127.7.39.62:34191
I20260811 12:18:54.749343 30463 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:54.750162 30463 heartbeater.cc:511] Master 127.7.39.62:34191 requested a full tablet report, sending...
I20260811 12:18:54.751572 29722 ts_manager.cc:194] Registered new tserver with Master: 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901)
I20260811 12:18:54.752370 29722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:55769
I20260811 12:18:54.763607 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:18:54.781502 30466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.768431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.7.39.62:34191)
0811 12:18:54.768608 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:54.768620 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:54.769006 (+ 386us) server_negotiation.cc:407] Connection header received
0811 12:18:54.769182 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:54.769187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:54.769273 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:54.769387 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:54.770892 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.771697 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.773196 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.773628 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.773747 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.776205 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:54.776210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:54.776213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:54.776252 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:54.778646 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:54.779383 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:54.779387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:54.779390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:54.779525 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:54.779916 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:54.779920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:54.779922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:54.780276 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:54.780512 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:54.780891 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 12:18:54.781143 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:18:54.785779 29722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:18:54.792897 29722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:18:54.818608 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.806913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33901 (local address 127.0.0.1:43284)
0811 12:18:54.807730 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:54.807784 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:54.807959 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:54.809334 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:54.809341 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:54.809374 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:54.809780 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.809795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.812027 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.812033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.814904 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.814928 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.817362 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.817370 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.817660 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.817733 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 12:18:54.817876 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 12:18:54.818152 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":327,"threads_started":1}
I20260811 12:18:54.819146 30488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.808153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48426)
0811 12:18:54.808671 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:54.808689 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:54.808802 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:54.810141 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:54.810147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:54.810179 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:54.810607 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.810618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.812986 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.812992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.815580 (+ 2588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.815602 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.818179 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.818187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.818427 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.818469 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 12:18:54.818540 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 12:18:54.818719 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":834,"threads_started":1}
I20260811 12:18:54.820236 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.807123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.7.39.3:33901)
0811 12:18:54.808683 (+ 1560us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:54.808690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:54.808719 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:18:54.808803 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:54.808809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:54.809012 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:54.809305 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:54.810001 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.811821 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.815221 (+ 3400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.816995 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.819261 (+ 2266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.819567 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:54.819656 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 12:18:54.819871 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1388,"thread_start_us":1236,"threads_started":1}
I20260811 12:18:54.820369 30489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.808794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.7.39.1:39543)
0811 12:18:54.809528 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:54.809536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:54.809563 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:18:54.809656 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:54.809663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:54.809911 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:54.810834 (+ 923us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:54.811141 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.812812 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.815866 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.817879 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.819465 (+ 1586us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.819741 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:54.819823 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 12:18:54.820013 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":799,"threads_started":1}
I20260811 12:18:54.825984 29907 tablet_service.cc:1467] Processing CreateTablet for tablet 274950ec1de3467c8f47f637375b051a (DEFAULT_TABLE table=test-table [id=73e4414b61a5415abe52f9c6e4a11a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:54.826805 30361 tablet_service.cc:1467] Processing CreateTablet for tablet 274950ec1de3467c8f47f637375b051a (DEFAULT_TABLE table=test-table [id=73e4414b61a5415abe52f9c6e4a11a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:54.827574 29907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 274950ec1de3467c8f47f637375b051a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:54.828353 30361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 274950ec1de3467c8f47f637375b051a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:54.828984 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.809965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:44745 (local address 127.0.0.1:48182)
0811 12:18:54.818908 (+ 8943us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:54.818930 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:54.819063 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:54.822702 (+ 3639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:54.822706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:54.822732 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:54.822877 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.822888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.824333 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.824338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.826251 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.826269 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.828184 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.828192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.828412 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.828450 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:18:54.828522 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:18:54.828679 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8818,"thread_start_us":11415,"threads_started":1}
I20260811 12:18:54.829756 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.810272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.7.39.2:44745)
0811 12:18:54.822008 (+ 11736us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:54.822018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:54.822052 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:18:54.822185 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:54.822192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:54.822473 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:54.822710 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:54.823020 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.824153 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.826503 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.827962 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.828739 (+ 777us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.828938 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:54.829026 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 12:18:54.829424 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11560,"thread_start_us":11434,"threads_started":1}
I20260811 12:18:54.833768 30135 tablet_service.cc:1467] Processing CreateTablet for tablet 274950ec1de3467c8f47f637375b051a (DEFAULT_TABLE table=test-table [id=73e4414b61a5415abe52f9c6e4a11a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:54.835335 30135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 274950ec1de3467c8f47f637375b051a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:54.840492 30493 tablet_bootstrap.cc:492] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Bootstrap starting.
I20260811 12:18:54.843487 30495 tablet_bootstrap.cc:492] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Bootstrap starting.
I20260811 12:18:54.845029 30493 tablet_bootstrap.cc:654] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:54.846525 30493 log.cc:824] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:54.847354 30495 tablet_bootstrap.cc:654] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:54.848714 30495 log.cc:824] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:54.849992 30498 tablet_bootstrap.cc:492] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Bootstrap starting.
I20260811 12:18:54.850095 30493 tablet_bootstrap.cc:492] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: No bootstrap required, opened a new log
I20260811 12:18:54.850412 30493 ts_tablet_manager.cc:1397] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:18:54.854246 30498 tablet_bootstrap.cc:654] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:54.856062 30498 log.cc:824] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:54.856463 30495 tablet_bootstrap.cc:492] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: No bootstrap required, opened a new log
I20260811 12:18:54.856814 30495 ts_tablet_manager.cc:1397] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260811 12:18:54.859040 30493 raft_consensus.cc:348] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.859426 30493 raft_consensus.cc:374] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:54.859520 30493 raft_consensus.cc:729] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d013cbbb9ad4920aadaebba30065933, State: Initialized, Role: FOLLOWER
I20260811 12:18:54.860021 30498 tablet_bootstrap.cc:492] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: No bootstrap required, opened a new log
I20260811 12:18:54.860167 30493 consensus_queue.cc:260] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.860405 30498 ts_tablet_manager.cc:1397] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20260811 12:18:54.862054 30493 ts_tablet_manager.cc:1428] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 12:18:54.867708 30495 raft_consensus.cc:348] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.868239 30495 raft_consensus.cc:374] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:54.868345 30495 raft_consensus.cc:729] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64bdd35fb68746bb83913ec4f840f35d, State: Initialized, Role: FOLLOWER
I20260811 12:18:54.869019 30495 consensus_queue.cc:260] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.870285 30463 heartbeater.cc:503] Master 127.7.39.62:34191 was elected leader, sending a full tablet report...
I20260811 12:18:54.871140 30495 ts_tablet_manager.cc:1428] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Time spent starting tablet: real 0.014s user 0.008s sys 0.008s
I20260811 12:18:54.871078 30498 raft_consensus.cc:348] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.871587 30498 raft_consensus.cc:374] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:54.871696 30498 raft_consensus.cc:729] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21705b13fc94aed9b598a38da54d6ba, State: Initialized, Role: FOLLOWER
I20260811 12:18:54.872313 30498 consensus_queue.cc:260] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.873451 30245 heartbeater.cc:503] Master 127.7.39.62:34191 was elected leader, sending a full tablet report...
I20260811 12:18:54.874006 30498 ts_tablet_manager.cc:1428] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260811 12:18:54.876495 30023 tablet.cc:2369] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:18:54.926877 30246 tablet.cc:2369] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:18:54.974829 30504 raft_consensus.cc:482] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:18:54.975142 30504 raft_consensus.cc:504] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.976931 30504 leader_election.cc:288] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901), b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745)
W20260811 12:18:54.977396 30465 tablet.cc:2369] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:18:54.984341 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.977402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:47527 (local address 127.7.39.3:33901)
0811 12:18:54.977654 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:54.977662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:54.978301 (+ 639us) server_negotiation.cc:407] Connection header received
0811 12:18:54.978663 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:54.978668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:54.978750 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:54.978816 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:54.979567 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.980609 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.982623 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.983392 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.983524 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.983632 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:54.983956 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:18:54.984099 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:18:54.984411 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.978413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:36655 (local address 127.7.39.2:44745)
0811 12:18:54.978623 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:54.978639 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:54.979006 (+ 367us) server_negotiation.cc:407] Connection header received
0811 12:18:54.979149 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:54.979153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:54.979214 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:54.979290 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:54.979883 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.980868 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.982742 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.983517 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.983648 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.983742 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:54.984037 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:18:54.984164 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:18:54.984549 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.978245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:44745 (local address 127.7.39.1:36655)
0811 12:18:54.978903 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:54.978927 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:54.979039 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:54.979337 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:54.979340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:54.979372 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:54.979732 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.979741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.981004 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.981008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.982585 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.982599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.983744 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.983751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.983893 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.983916 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:18:54.983995 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:18:54.984126 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":6,"thread_start_us":325,"threads_started":1}
I20260811 12:18:54.984889 30511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:54.977387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33901 (local address 127.7.39.1:47527)
0811 12:18:54.978179 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:54.978201 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:54.978460 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:54.979033 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:54.979038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:54.979066 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:54.979409 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.979424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.980755 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.980761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.982463 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:54.982476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:54.983611 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:54.983618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:54.983804 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:54.983830 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:18:54.983913 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:18:54.984076 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":20,"thread_start_us":432,"threads_started":1}
I20260811 12:18:54.985502 30162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "274950ec1de3467c8f47f637375b051a" candidate_uuid: "3d013cbbb9ad4920aadaebba30065933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21705b13fc94aed9b598a38da54d6ba" is_pre_election: true
I20260811 12:18:54.985720 30390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "274950ec1de3467c8f47f637375b051a" candidate_uuid: "3d013cbbb9ad4920aadaebba30065933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdd35fb68746bb83913ec4f840f35d" is_pre_election: true
I20260811 12:18:54.986011 30162 raft_consensus.cc:2454] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d013cbbb9ad4920aadaebba30065933 in term 0.
I20260811 12:18:54.986209 30390 raft_consensus.cc:2454] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d013cbbb9ad4920aadaebba30065933 in term 0.
I20260811 12:18:54.986646 29843 leader_election.cc:302] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d013cbbb9ad4920aadaebba30065933, b21705b13fc94aed9b598a38da54d6ba; no voters:
I20260811 12:18:54.987039 30504 raft_consensus.cc:2781] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:18:54.987192 30504 raft_consensus.cc:482] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:18:54.987308 30504 raft_consensus.cc:3037] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:54.989635 30504 raft_consensus.cc:504] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.990243 30504 leader_election.cc:288] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 1 election: Requested vote from peers 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901), b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745)
I20260811 12:18:54.990739 30390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "274950ec1de3467c8f47f637375b051a" candidate_uuid: "3d013cbbb9ad4920aadaebba30065933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdd35fb68746bb83913ec4f840f35d"
I20260811 12:18:54.990819 30162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "274950ec1de3467c8f47f637375b051a" candidate_uuid: "3d013cbbb9ad4920aadaebba30065933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21705b13fc94aed9b598a38da54d6ba"
I20260811 12:18:54.990921 30390 raft_consensus.cc:3037] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:54.991062 30162 raft_consensus.cc:3037] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:54.993112 30390 raft_consensus.cc:2454] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d013cbbb9ad4920aadaebba30065933 in term 1.
I20260811 12:18:54.993273 30162 raft_consensus.cc:2454] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d013cbbb9ad4920aadaebba30065933 in term 1.
I20260811 12:18:54.993572 29843 leader_election.cc:302] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d013cbbb9ad4920aadaebba30065933, 64bdd35fb68746bb83913ec4f840f35d; no voters:
I20260811 12:18:54.993848 30504 raft_consensus.cc:2781] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:18:54.994249 30504 raft_consensus.cc:686] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 1 LEADER]: Becoming Leader. State: Replica: 3d013cbbb9ad4920aadaebba30065933, State: Running, Role: LEADER
I20260811 12:18:54.994736 30504 consensus_queue.cc:237] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } }
I20260811 12:18:54.999682 29722 catalog_manager.cc:5495] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d013cbbb9ad4920aadaebba30065933 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "3d013cbbb9ad4920aadaebba30065933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } health_report { overall_health: UNKNOWN } } }
I20260811 12:18:55.030537 30489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:55.025496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.7.39.1:39543)
0811 12:18:55.025696 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:55.025701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:55.025720 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:18:55.025848 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:55.025852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:55.025953 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:55.026094 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:18:55.026400 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:55.027145 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:55.028546 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:55.028976 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:55.029073 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:55.029558 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:18:55.030101 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:18:55.030216 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:55.030258 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 12:18:55.030330 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:18:55.038924 30162 raft_consensus.cc:1261] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 1 FOLLOWER]: Refusing update from remote peer 3d013cbbb9ad4920aadaebba30065933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:18:55.038928 30390 raft_consensus.cc:1261] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 1 FOLLOWER]: Refusing update from remote peer 3d013cbbb9ad4920aadaebba30065933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:18:55.039894 30514 consensus_queue.cc:1035] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:18:55.040282 30504 consensus_queue.cc:1035] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:18:55.066593 29722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:18:55.066855 29722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:18:55.070717 29722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Sending DeleteTablet for 3 replicas of tablet 274950ec1de3467c8f47f637375b051a
I20260811 12:18:55.072119 30361 tablet_service.cc:1514] Processing DeleteTablet for tablet 274950ec1de3467c8f47f637375b051a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:55 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:43284
I20260811 12:18:55.072331 30135 tablet_service.cc:1514] Processing DeleteTablet for tablet 274950ec1de3467c8f47f637375b051a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:55 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:48182
I20260811 12:18:55.072388 29907 tablet_service.cc:1514] Processing DeleteTablet for tablet 274950ec1de3467c8f47f637375b051a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:55 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:48426
I20260811 12:18:55.073729 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 29818
I20260811 12:18:55.083853 30544 tablet_replica.cc:330] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: stopping tablet replica
I20260811 12:18:55.084666 30544 raft_consensus.cc:2229] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:18:55.085664 30544 raft_consensus.cc:2258] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:55.086834 30543 tablet_replica.cc:330] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: stopping tablet replica
I20260811 12:18:55.087387 30543 raft_consensus.cc:2229] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:18:55.088009 30543 raft_consensus.cc:2258] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:55.088789 30544 ts_tablet_manager.cc:1905] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:55.090340 30543 ts_tablet_manager.cc:1905] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:55.094170 30542 tablet_replica.cc:330] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: stopping tablet replica
I20260811 12:18:55.094683 30542 raft_consensus.cc:2229] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:18:55.095227 30542 raft_consensus.cc:2258] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:55.098359 30543 ts_tablet_manager.cc:1918] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:18:55.098608 30543 log.cc:1196] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/274950ec1de3467c8f47f637375b051a
I20260811 12:18:55.099269 30543 ts_tablet_manager.cc:1939] T 274950ec1de3467c8f47f637375b051a P b21705b13fc94aed9b598a38da54d6ba: Deleting consensus metadata
I20260811 12:18:55.100463 30542 ts_tablet_manager.cc:1905] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:55.100821 30544 ts_tablet_manager.cc:1918] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:18:55.101037 30544 log.cc:1196] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/274950ec1de3467c8f47f637375b051a
I20260811 12:18:55.101370 29697 catalog_manager.cc:4841] TS b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745): tablet 274950ec1de3467c8f47f637375b051a (table test-table [id=73e4414b61a5415abe52f9c6e4a11a1a]) successfully deleted
I20260811 12:18:55.101734 30544 ts_tablet_manager.cc:1939] T 274950ec1de3467c8f47f637375b051a P 3d013cbbb9ad4920aadaebba30065933: Deleting consensus metadata
Aug 11 12:18:55 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:18:55.106352 30542 ts_tablet_manager.cc:1918] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:18:55.106549 30542 log.cc:1196] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/274950ec1de3467c8f47f637375b051a
I20260811 12:18:55.106846 29697 catalog_manager.cc:4841] TS 3d013cbbb9ad4920aadaebba30065933 (127.7.39.1:39543): tablet 274950ec1de3467c8f47f637375b051a (table test-table [id=73e4414b61a5415abe52f9c6e4a11a1a]) successfully deleted
I20260811 12:18:55.107126 30542 ts_tablet_manager.cc:1939] T 274950ec1de3467c8f47f637375b051a P 64bdd35fb68746bb83913ec4f840f35d: Deleting consensus metadata
I20260811 12:18:55.108870 29697 catalog_manager.cc:4841] TS 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901): tablet 274950ec1de3467c8f47f637375b051a (table test-table [id=73e4414b61a5415abe52f9c6e4a11a1a]) successfully deleted
I20260811 12:18:55.115684 30489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:55.074990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48450 (local address 127.7.39.1:39543)
0811 12:18:55.075184 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:55.075191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:55.091649 (+ 16458us) server_negotiation.cc:407] Connection header received
0811 12:18:55.092217 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:55.092224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:55.092332 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:55.092412 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:55.094784 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:55.096012 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:55.098260 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:55.100643 (+ 2383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:55.100780 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:55.107967 (+ 7187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:55.107984 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:55.108005 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:55.108061 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:55.112312 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:55.113213 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:55.113228 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:55.113232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:55.113452 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:55.113865 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:55.113870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:55.113874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:55.114412 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:55.114743 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:55.115030 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 12:18:55.115302 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:18:55.116516 30016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:55.218178 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 29818
I20260811 12:18:55.242085 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 30028
Aug 11 12:18:55 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:18:55.257700 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:55.242913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.7.39.2:44745)
0811 12:18:55.243090 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:55.243094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:55.243113 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:18:55.243210 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:55.243214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:55.243292 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:55.243381 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:55.244766 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:55.245596 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:55.247034 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:55.247428 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:55.247524 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:55.251887 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:55.251905 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:55.251923 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:55.251962 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:55.254695 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:55.255496 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:55.255505 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:55.255507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:55.255661 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:55.256044 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:55.256047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:55.256050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:55.256525 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:55.256765 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:55.257022 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 12:18:55.257366 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:18:55.258476 30237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:55.351089 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 30028
I20260811 12:18:55.378528 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 30253
Aug 11 12:18:55 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:18:55.394104 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:55.379269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43292 (local address 127.7.39.3:33901)
0811 12:18:55.379429 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:55.379434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:55.379454 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:18:55.379623 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:55.379646 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:55.379714 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:55.379799 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:55.381085 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:55.381861 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:55.383191 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:55.383581 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:55.383680 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:55.388324 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:55.388343 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:55.388361 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:55.388405 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:55.391271 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:55.392003 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:55.392010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:55.392013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:55.392167 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:55.392531 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:55.392536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:55.392538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:55.392965 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:55.393341 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:55.393630 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 12:18:55.393819 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:18:55.394788 30456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:55.483788 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 30253
I20260811 12:18:55.506692 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 29667
I20260811 12:18:55.508014 29768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:55.619834 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 29667
I20260811 12:18:55.646735 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34191
--webserver_interface=127.7.39.62
--webserver_port=34439
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:56.010663 30558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:56.011008 30558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:56.011153 30558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:56.018633 30558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:56.018767 30558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:56.018852 30558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:56.018929 30558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:56.018991 30558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:56.029353 30558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=34439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:56.030767 30558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:56.032228 30558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:56.059242 30570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:56.059173 30571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:56.059217 30577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:56.059965 30558 server_base.cc:1034] running on GCE node
I20260811 12:18:56.060889 30558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:56.062608 30558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:56.063835 30558 hybrid_clock.cc:648] HybridClock initialized: now 1786450736063763 us; error 54 us; skew 500 ppm
I20260811 12:18:56.064246 30558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:56 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:56.067991 30558 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:18:56.069911 30558 webserver.cc:460] Webserver started at http://127.7.39.62:34439/ using document root <none> and password file <none>
I20260811 12:18:56.070739 30558 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:56.070899 30558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:56.077354 30558 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 12:18:56.080543 30585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:56.082099 30558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:18:56.082289 30558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "92995140ec2b42e6af70790eb20f5953"
format_stamp: "Formatted at 2026-08-11 12:18:52 on dist-test-slave-rmhg"
server_key: "6b9ed4016c1b98dc33235447974ea044"
server_key_iv: "60b704c762d50778b2c558f65137d7a9"
server_key_version: "encryptionkey@0"
I20260811 12:18:56.083427 30558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:56.096189 30558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:56.101377 30558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:56.101697 30558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:56.122196 30558 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:34191
I20260811 12:18:56.122215 30664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:34191 every 8 connection(s)
I20260811 12:18:56.123919 30558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:18:56.125991 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 30558
I20260811 12:18:56.126812 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:39543
--local_ip_for_outbound_sockets=127.7.39.1
--tserver_master_addrs=127.7.39.62:34191
--webserver_port=39535
--webserver_interface=127.7.39.1
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:18:56.131636 30665 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:18:56.135366 30665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Bootstrap starting.
I20260811 12:18:56.155938 30665 log.cc:824] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:56.169516 30665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:18:56.170324 30665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Bootstrap complete.
I20260811 12:18:56.180142 30665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:56.181108 30665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92995140ec2b42e6af70790eb20f5953, State: Initialized, Role: FOLLOWER
I20260811 12:18:56.181828 30665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:56.182090 30665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:18:56.182209 30665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:18:56.182375 30665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:18:56.186684 30665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:56.187314 30665 leader_election.cc:302] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92995140ec2b42e6af70790eb20f5953; no voters:
I20260811 12:18:56.187988 30665 leader_election.cc:288] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:18:56.188144 30678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:18:56.190485 30665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:56.195264 30678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 2 LEADER]: Becoming Leader. State: Replica: 92995140ec2b42e6af70790eb20f5953, State: Running, Role: LEADER
I20260811 12:18:56.196147 30678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:56.199072 30679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "92995140ec2b42e6af70790eb20f5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } } }
I20260811 12:18:56.199523 30679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:56.200126 30680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92995140ec2b42e6af70790eb20f5953. Latest consensus state: current_term: 2 leader_uuid: "92995140ec2b42e6af70790eb20f5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } } }
I20260811 12:18:56.200413 30680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:56.200457 30692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:18:56.220479 30692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:18:56.222119 30692 catalog_manager.cc:1259] Loaded cluster ID: 860831a079694580aa8a3fab1a01286d
I20260811 12:18:56.222314 30692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:18:56.225895 30692 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 12:18:56.246802 30692 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 12:18:56.250440 30708 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:18:56.258370 30692 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:18:56.260174 30692 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Loaded TSK: 0
I20260811 12:18:56.261855 30692 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:18:56.504441 30671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:56.504817 30671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:56.504951 30671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:56.512727 30671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:56.512859 30671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:56.513005 30671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:18:56.523289 30671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:39543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=39535
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:56.524757 30671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:56.526343 30671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:56.541674 30720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:56.542088 30716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:56.541792 30718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:56.543167 30671 server_base.cc:1034] running on GCE node
I20260811 12:18:56.544052 30671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:56.545506 30671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:56.546800 30671 hybrid_clock.cc:648] HybridClock initialized: now 1786450736546695 us; error 68 us; skew 500 ppm
I20260811 12:18:56.547230 30671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:56 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:56.551324 30671 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:18:56.553191 30671 webserver.cc:460] Webserver started at http://127.7.39.1:39535/ using document root <none> and password file <none>
I20260811 12:18:56.554117 30671 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:56.554289 30671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:56.561975 30671 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:18:56.566092 30731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:56.567778 30671 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:18:56.567981 30671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "3d013cbbb9ad4920aadaebba30065933"
format_stamp: "Formatted at 2026-08-11 12:18:53 on dist-test-slave-rmhg"
server_key: "c6a7388778ecc2aaba598d55d4c607f3"
server_key_iv: "026c86de5a793cb58599930e00a2c0e1"
server_key_version: "encryptionkey@0"
I20260811 12:18:56.569119 30671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:56.590981 30671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:56.595695 30671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:56.595965 30671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:56.597122 30671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:56.598549 30671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:56.598660 30671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:56.598779 30671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:56.598850 30671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:56.630041 30671 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:39543
I20260811 12:18:56.630116 30921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:39543 every 8 connection(s)
I20260811 12:18:56.631518 30671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:18:56.637715 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 30671
I20260811 12:18:56.638401 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:44745
--local_ip_for_outbound_sockets=127.7.39.2
--tserver_master_addrs=127.7.39.62:34191
--webserver_port=35601
--webserver_interface=127.7.39.2
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:18:56 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:56.658578 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:56.634759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:53851)
0811 12:18:56.635573 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:56.635613 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:56.637508 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:56.639133 (+ 1625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:56.639144 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:56.639896 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:56.640670 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:56.640696 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:56.642542 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:56.642548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:56.644415 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:56.644449 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:56.645784 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:56.645792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:56.646057 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:56.646836 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:56.646873 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:56.649667 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:56.654038 (+ 4371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:56.654046 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:56.654061 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:56.654522 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:56.655066 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:56.655069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:56.655071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:56.655273 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:56.656287 (+ 1014us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:56.656294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:56.656644 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 12:18:56.656892 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 12:18:56.657389 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":342,"threads_started":1}
I20260811 12:18:56.658947 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:56.634493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:53851 (local address 127.7.39.62:34191)
0811 12:18:56.635106 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:56.635124 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:56.635762 (+ 638us) server_negotiation.cc:407] Connection header received
0811 12:18:56.638140 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:56.638155 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:56.638550 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:56.638909 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:56.640927 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:56.642336 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:56.644627 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:56.645508 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:56.645693 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:56.650022 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:56.650037 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:56.650061 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:56.650118 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:56.653846 (+ 3728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:56.654697 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:56.654707 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:56.654716 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:56.654959 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:56.655397 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:56.655402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:56.655406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:56.656155 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:56.656374 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:56.657185 (+ 811us) server_negotiation.cc:299] Negotiation successful
0811 12:18:56.657739 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":272,"threads_started":1}
I20260811 12:18:56.662108 30924 heartbeater.cc:347] Connected to a master server at 127.7.39.62:34191
I20260811 12:18:56.662478 30924 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:56.663307 30924 heartbeater.cc:511] Master 127.7.39.62:34191 requested a full tablet report, sending...
I20260811 12:18:56.666190 30610 ts_manager.cc:194] Registered new tserver with Master: 3d013cbbb9ad4920aadaebba30065933 (127.7.39.1:39543)
I20260811 12:18:56.668382 30610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:53851
W20260811 12:18:56.993463 30937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:56.993779 30937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:56.993885 30937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:57.001330 30937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:57.001456 30937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:57.001583 30937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:18:57.011742 30937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:44745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=35601
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:57.013098 30937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:57.014497 30937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:57.027860 30945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:57.027881 30946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:57.027923 30948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:57.028688 30937 server_base.cc:1034] running on GCE node
I20260811 12:18:57.029347 30937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:57.030504 30937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:57.031729 30937 hybrid_clock.cc:648] HybridClock initialized: now 1786450737031666 us; error 54 us; skew 500 ppm
I20260811 12:18:57.032083 30937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:57 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450737, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:57.035300 30937 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:18:57.036804 30937 webserver.cc:460] Webserver started at http://127.7.39.2:35601/ using document root <none> and password file <none>
I20260811 12:18:57.037595 30937 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:57.037738 30937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:57.043429 30937 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:18:57.046502 30959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:57.047935 30937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:18:57.048132 30937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "b21705b13fc94aed9b598a38da54d6ba"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "43bfbbd88f831912b87a3755985e477b"
server_key_iv: "cd733863f780e11a35d60824342ac0e6"
server_key_version: "encryptionkey@0"
I20260811 12:18:57.049197 30937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:57.079879 30937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:57.084233 30937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:57.084549 30937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:57.085637 30937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:57.087172 30937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:57.087278 30937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:57.087388 30937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:57.087458 30937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:57.120054 30937 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:44745
I20260811 12:18:57.120133 31140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:44745 every 8 connection(s)
I20260811 12:18:57.121650 30937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:18:57.123596 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 30937
I20260811 12:18:57.124280 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:33901
--local_ip_for_outbound_sockets=127.7.39.3
--tserver_master_addrs=127.7.39.62:34191
--webserver_port=38963
--webserver_interface=127.7.39.3
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:18:57 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450737, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:57.148633 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.124843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:41719 (local address 127.7.39.62:34191)
0811 12:18:57.125042 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.125056 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.128159 (+ 3103us) server_negotiation.cc:407] Connection header received
0811 12:18:57.130994 (+ 2835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.131001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.131122 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.131238 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:57.133435 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.134400 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.136906 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.137623 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.137752 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.142373 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:57.142379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:57.142382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:57.142420 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:57.144917 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:57.145966 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:57.145971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:57.145973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:57.146140 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:57.146756 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:57.146760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:57.146762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:57.147086 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:57.147208 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.148156 (+ 948us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.148363 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:18:57.150291 31146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.125411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:41719)
0811 12:18:57.127917 (+ 2506us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:57.127985 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:57.130599 (+ 2614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:57.131558 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:57.131571 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:57.132385 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:57.133217 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.133244 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.134615 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.134621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.136710 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.136726 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.137886 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.137894 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.138220 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.139004 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:57.139034 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:57.142109 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:57.145106 (+ 2997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:57.145116 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:57.145136 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:57.145775 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:57.146285 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:57.146291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:57.146294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:57.146614 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:57.147224 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:57.147234 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:57.147727 (+ 493us) client_negotiation.cc:769] Sending connection context
0811 12:18:57.148049 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 12:18:57.148692 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2167,"thread_start_us":1938,"threads_started":1}
I20260811 12:18:57.152120 31141 heartbeater.cc:347] Connected to a master server at 127.7.39.62:34191
I20260811 12:18:57.152449 31141 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:57.153358 31141 heartbeater.cc:511] Master 127.7.39.62:34191 requested a full tablet report, sending...
I20260811 12:18:57.155056 30610 ts_manager.cc:194] Registered new tserver with Master: b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745)
I20260811 12:18:57.156242 30610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:41719
W20260811 12:18:57.478097 31148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:57.478399 31148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:57.478503 31148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:57.486279 31148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:57.486429 31148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:57.486554 31148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:18:57.497542 31148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:33901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=38963
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:57.498953 31148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:57.500371 31148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:57.514621 31162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:57.514551 31160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:57.514596 31166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:57.515499 31148 server_base.cc:1034] running on GCE node
I20260811 12:18:57.516149 31148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:57.517246 31148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:57.518458 31148 hybrid_clock.cc:648] HybridClock initialized: now 1786450737518390 us; error 65 us; skew 500 ppm
I20260811 12:18:57.518793 31148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:57 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450737, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:57.522075 31148 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:18:57.523663 31148 webserver.cc:460] Webserver started at http://127.7.39.3:38963/ using document root <none> and password file <none>
I20260811 12:18:57.524461 31148 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:57.524605 31148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:57.530615 31148 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:18:57.533696 31180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:57.535292 31148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:18:57.535480 31148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "64bdd35fb68746bb83913ec4f840f35d"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "fdf69fb0e8aa44886ee820bd4ef7f2d1"
server_key_iv: "11fccc9fe940e3493d570233580be3f7"
server_key_version: "encryptionkey@0"
I20260811 12:18:57.536649 31148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:57.552350 31148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:57.557124 31148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:57.557427 31148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:57.558449 31148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:57.559877 31148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:18:57.559973 31148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:57.560056 31148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:18:57.560123 31148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:57.593106 31148 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:33901
I20260811 12:18:57.593195 31361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:33901 every 8 connection(s)
I20260811 12:18:57.594687 31148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:18:57.597702 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 31148
Aug 11 12:18:57 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450737, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:18:57.615100 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.598789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.7.39.62:34191)
0811 12:18:57.600500 (+ 1711us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.600506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.600524 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:18:57.600585 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.600590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.600680 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.600792 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:57.602370 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.603493 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.605005 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.605665 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.605795 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.608966 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:57.608971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:57.608974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:57.609004 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:57.612223 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:57.613030 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:57.613034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:57.613036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:57.613232 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:57.613614 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:57.613619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:57.613622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:57.614022 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:57.614327 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.614541 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.614770 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1591,"thread_start_us":1468,"threads_started":1}
I20260811 12:18:57.618235 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.597349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:50975 (local address 127.7.39.62:34191)
0811 12:18:57.597531 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.597536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.599215 (+ 1679us) server_negotiation.cc:407] Connection header received
0811 12:18:57.601684 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.601689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.601762 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.601842 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:57.603822 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.604556 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.606569 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.607059 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.607181 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.611429 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:57.611436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:57.611444 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:57.611504 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:57.614603 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:57.615578 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:57.615582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:57.615584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:57.615722 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:57.616287 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:57.616292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:57.616295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:57.616637 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:57.616779 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.617657 (+ 878us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.617892 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:18:57.619256 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.598099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:50975)
0811 12:18:57.599028 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:57.599077 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:57.601404 (+ 2327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:57.602100 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:57.602109 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:57.602848 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:18:57.603669 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.603693 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.604712 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.604715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.606433 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.606445 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.607334 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.607342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.607666 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.608396 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:18:57.608428 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:18:57.611166 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:18:57.614804 (+ 3638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:57.614815 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:57.614837 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:57.615443 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:57.615846 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:18:57.615850 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:18:57.615853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:18:57.616159 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:18:57.616765 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:18:57.616773 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:18:57.617202 (+ 429us) client_negotiation.cc:769] Sending connection context
0811 12:18:57.617487 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 12:18:57.618068 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":352,"threads_started":1}
I20260811 12:18:57.620645 31362 heartbeater.cc:347] Connected to a master server at 127.7.39.62:34191
I20260811 12:18:57.620985 31362 heartbeater.cc:464] Registering TS with master...
I20260811 12:18:57.621892 31362 heartbeater.cc:511] Master 127.7.39.62:34191 requested a full tablet report, sending...
I20260811 12:18:57.623579 30610 ts_manager.cc:194] Registered new tserver with Master: 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901)
I20260811 12:18:57.624439 30610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:50975
I20260811 12:18:57.630775 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:18:57.637936 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.631728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.7.39.62:34191)
0811 12:18:57.631886 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.631890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.632438 (+ 548us) server_negotiation.cc:407] Connection header received
0811 12:18:57.632693 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.632697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.632833 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.632969 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:18:57.633496 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.634271 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.635744 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.636149 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.636259 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.636808 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:18:57.637369 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:18:57.637516 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.637614 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.637701 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:18:57.638864 30610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:18:57.646705 30610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:18:57.672240 30924 heartbeater.cc:503] Master 127.7.39.62:34191 was elected leader, sending a full tablet report...
I20260811 12:18:57.674576 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.658640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:44745 (local address 127.0.0.1:48208)
0811 12:18:57.659940 (+ 1300us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:57.659985 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:57.660162 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:57.661172 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:57.661182 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:57.661219 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:57.661644 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.661658 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.665321 (+ 3663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.665328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.669673 (+ 4345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.669696 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.673073 (+ 3377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.673080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.673373 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.673438 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 12:18:57.673949 (+ 511us) client_negotiation.cc:240] Negotiation successful
0811 12:18:57.674194 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"thread_start_us":957,"threads_started":1}
I20260811 12:18:57.675105 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.658930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.7.39.2:44745)
0811 12:18:57.659531 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.659541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.660094 (+ 553us) server_negotiation.cc:407] Connection header received
0811 12:18:57.660355 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.660361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.660627 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.660826 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:57.663216 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.665081 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.669944 (+ 4863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.672460 (+ 2516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.672612 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.672836 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.673701 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.674749 (+ 1048us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":305,"threads_started":1}
I20260811 12:18:57.678436 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.660306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33901 (local address 127.0.0.1:43306)
0811 12:18:57.660864 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:57.660882 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:57.660994 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:57.671558 (+ 10564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:57.671564 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:57.671601 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:57.672069 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.672084 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.674057 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.674062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.676071 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.676086 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.677559 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.677565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.677763 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.677788 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 12:18:57.677874 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:18:57.678024 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"spinlock_wait_cycles":1408,"thread_start_us":287,"threads_started":1}
I20260811 12:18:57.678448 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.660678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.7.39.3:33901)
0811 12:18:57.670562 (+ 9884us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.670572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.670613 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:18:57.670731 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.670740 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.671030 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.671267 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:57.672281 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.673872 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.676244 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.677332 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.677472 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.677711 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.677948 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.678106 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9678,"thread_start_us":9510,"threads_started":1}
I20260811 12:18:57.679643 31038 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4eac8793cf4037bb9abe42faad9511 (DEFAULT_TABLE table=test-table [id=60771b2268c145df8df506568fb03059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:57.680862 31387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.660909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48458)
0811 12:18:57.669468 (+ 8559us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:57.669503 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:57.669678 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:57.671558 (+ 1880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:57.671564 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:57.671610 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:57.672717 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.672734 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.674837 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.674842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.678122 (+ 3280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.678139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.680001 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.680009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.680180 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.680214 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:18:57.680322 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 12:18:57.680460 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8366,"mutex_wait_us":2,"spinlock_wait_cycles":3456,"thread_start_us":8881,"threads_started":1}
I20260811 12:18:57.680904 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.661494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.7.39.1:39543)
0811 12:18:57.670963 (+ 9469us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.670972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.671001 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:18:57.671112 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.671121 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.671350 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.671576 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:57.672970 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.674685 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.678322 (+ 3637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.679762 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.679907 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.680126 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.680381 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.680559 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9289,"thread_start_us":9133,"threads_started":1}
I20260811 12:18:57.681396 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4eac8793cf4037bb9abe42faad9511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:57.682734 31260 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4eac8793cf4037bb9abe42faad9511 (DEFAULT_TABLE table=test-table [id=60771b2268c145df8df506568fb03059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:57.683982 31260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4eac8793cf4037bb9abe42faad9511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:57.684556 30814 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4eac8793cf4037bb9abe42faad9511 (DEFAULT_TABLE table=test-table [id=60771b2268c145df8df506568fb03059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:18:57.685747 30814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4eac8793cf4037bb9abe42faad9511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:18:57.696573 31390 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Bootstrap starting.
I20260811 12:18:57.700651 31390 tablet_bootstrap.cc:654] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:57.700896 31392 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Bootstrap starting.
I20260811 12:18:57.702229 31390 log.cc:824] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:57.703264 31394 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Bootstrap starting.
I20260811 12:18:57.704849 31392 tablet_bootstrap.cc:654] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:57.706254 31392 log.cc:824] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:57.706652 31394 tablet_bootstrap.cc:654] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Neither blocks nor log segments found. Creating new log.
I20260811 12:18:57.707291 31390 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: No bootstrap required, opened a new log
I20260811 12:18:57.707563 31390 ts_tablet_manager.cc:1397] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 12:18:57.707902 31394 log.cc:824] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:57.709728 31392 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: No bootstrap required, opened a new log
I20260811 12:18:57.710057 31392 ts_tablet_manager.cc:1397] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260811 12:18:57.715587 31394 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: No bootstrap required, opened a new log
I20260811 12:18:57.715869 31394 ts_tablet_manager.cc:1397] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260811 12:18:57.718257 31390 raft_consensus.cc:348] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.718758 31390 raft_consensus.cc:374] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:57.718918 31390 raft_consensus.cc:729] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64bdd35fb68746bb83913ec4f840f35d, State: Initialized, Role: FOLLOWER
I20260811 12:18:57.719643 31390 consensus_queue.cc:260] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.720834 31362 heartbeater.cc:503] Master 127.7.39.62:34191 was elected leader, sending a full tablet report...
I20260811 12:18:57.720639 31392 raft_consensus.cc:348] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.721084 31392 raft_consensus.cc:374] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:57.721244 31392 raft_consensus.cc:729] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d013cbbb9ad4920aadaebba30065933, State: Initialized, Role: FOLLOWER
I20260811 12:18:57.721638 31390 ts_tablet_manager.cc:1428] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260811 12:18:57.721901 31392 consensus_queue.cc:260] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.726493 31392 ts_tablet_manager.cc:1428] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260811 12:18:57.726691 31394 raft_consensus.cc:348] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.727171 31394 raft_consensus.cc:374] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:18:57.727274 31394 raft_consensus.cc:729] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21705b13fc94aed9b598a38da54d6ba, State: Initialized, Role: FOLLOWER
I20260811 12:18:57.727869 31394 consensus_queue.cc:260] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.728938 31141 heartbeater.cc:503] Master 127.7.39.62:34191 was elected leader, sending a full tablet report...
I20260811 12:18:57.729666 31394 ts_tablet_manager.cc:1428] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
W20260811 12:18:57.846400 31364 tablet.cc:2369] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:18:57.873732 31144 tablet.cc:2369] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:18:57.883955 30930 tablet.cc:2369] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:18:57.945833 31405 raft_consensus.cc:482] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:18:57.946153 31405 raft_consensus.cc:504] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.947846 31405 leader_election.cc:288] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901), 3d013cbbb9ad4920aadaebba30065933 (127.7.39.1:39543)
I20260811 12:18:57.954818 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.948414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:45005 (local address 127.7.39.3:33901)
0811 12:18:57.948596 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.948601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.949181 (+ 580us) server_negotiation.cc:407] Connection header received
0811 12:18:57.949456 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.949462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.949544 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.949608 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:57.950312 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.951067 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.952905 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.953796 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.953911 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.954015 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.954380 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.954508 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:18:57.955041 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.949385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:56565 (local address 127.7.39.1:39543)
0811 12:18:57.949601 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:57.949607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:57.949809 (+ 202us) server_negotiation.cc:407] Connection header received
0811 12:18:57.950047 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:57.950052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:57.950119 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:57.950197 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:18:57.950709 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.951428 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.953216 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.954180 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.954300 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.954400 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:57.954685 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 12:18:57.954809 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:18:57.955170 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.949114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.7.39.2:56565)
0811 12:18:57.949696 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:57.949712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:57.949907 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:57.950236 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:57.950240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:57.950266 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:57.950552 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.950560 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.951575 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.951579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.953047 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.953060 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.954392 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.954398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.954536 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.954560 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:18:57.954651 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:18:57.954786 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"spinlock_wait_cycles":24960,"thread_start_us":462,"threads_started":1}
I20260811 12:18:57.955687 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:57.948286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33901 (local address 127.7.39.2:45005)
0811 12:18:57.949037 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:57.949057 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:18:57.949238 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:18:57.949818 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:18:57.949822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:18:57.949856 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:18:57.950166 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.950174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.951200 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.951204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.952732 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:18:57.952745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:18:57.954009 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:18:57.954016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:18:57.954182 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:57.954215 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:18:57.955282 (+ 1067us) client_negotiation.cc:240] Negotiation successful
0811 12:18:57.955428 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":410,"threads_started":1}
I20260811 12:18:57.956269 30845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4eac8793cf4037bb9abe42faad9511" candidate_uuid: "b21705b13fc94aed9b598a38da54d6ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013cbbb9ad4920aadaebba30065933" is_pre_election: true
I20260811 12:18:57.956674 31290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4eac8793cf4037bb9abe42faad9511" candidate_uuid: "b21705b13fc94aed9b598a38da54d6ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdd35fb68746bb83913ec4f840f35d" is_pre_election: true
I20260811 12:18:57.956794 30845 raft_consensus.cc:2454] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b21705b13fc94aed9b598a38da54d6ba in term 0.
I20260811 12:18:57.957137 31290 raft_consensus.cc:2454] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b21705b13fc94aed9b598a38da54d6ba in term 0.
I20260811 12:18:57.957484 30963 leader_election.cc:302] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d013cbbb9ad4920aadaebba30065933, b21705b13fc94aed9b598a38da54d6ba; no voters:
I20260811 12:18:57.957957 31405 raft_consensus.cc:2781] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:18:57.958094 31405 raft_consensus.cc:482] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:18:57.958212 31405 raft_consensus.cc:3037] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:57.960420 31405 raft_consensus.cc:504] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.960996 31405 leader_election.cc:288] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [CANDIDATE]: Term 1 election: Requested vote from peers 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901), 3d013cbbb9ad4920aadaebba30065933 (127.7.39.1:39543)
I20260811 12:18:57.961541 31290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4eac8793cf4037bb9abe42faad9511" candidate_uuid: "b21705b13fc94aed9b598a38da54d6ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdd35fb68746bb83913ec4f840f35d"
I20260811 12:18:57.961616 30845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4eac8793cf4037bb9abe42faad9511" candidate_uuid: "b21705b13fc94aed9b598a38da54d6ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013cbbb9ad4920aadaebba30065933"
I20260811 12:18:57.961719 31290 raft_consensus.cc:3037] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:57.961825 30845 raft_consensus.cc:3037] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:18:57.963923 31290 raft_consensus.cc:2454] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b21705b13fc94aed9b598a38da54d6ba in term 1.
I20260811 12:18:57.964033 30845 raft_consensus.cc:2454] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b21705b13fc94aed9b598a38da54d6ba in term 1.
I20260811 12:18:57.964339 30963 leader_election.cc:302] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64bdd35fb68746bb83913ec4f840f35d, b21705b13fc94aed9b598a38da54d6ba; no voters:
I20260811 12:18:57.964686 31405 raft_consensus.cc:2781] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:18:57.965178 31405 raft_consensus.cc:686] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 1 LEADER]: Becoming Leader. State: Replica: b21705b13fc94aed9b598a38da54d6ba, State: Running, Role: LEADER
I20260811 12:18:57.965662 31405 consensus_queue.cc:237] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:57.970252 30609 catalog_manager.cc:5495] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba reported cstate change: term changed from 0 to 1, leader changed from <none> to b21705b13fc94aed9b598a38da54d6ba (127.7.39.2). New cstate: current_term: 1 leader_uuid: "b21705b13fc94aed9b598a38da54d6ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } health_report { overall_health: UNKNOWN } } }
I20260811 12:18:58.038656 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.033539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.7.39.2:44745)
0811 12:18:58.033720 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:58.033725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:58.033742 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:18:58.033874 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:58.033877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:58.033957 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:58.034085 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:18:58.034374 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:58.035078 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:58.036460 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:58.036852 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:58.036939 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:58.037502 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:18:58.038027 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:18:58.038192 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:58.038291 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 12:18:58.038380 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:18:58.047600 31290 raft_consensus.cc:1261] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 1 FOLLOWER]: Refusing update from remote peer b21705b13fc94aed9b598a38da54d6ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:18:58.047616 30845 raft_consensus.cc:1261] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Refusing update from remote peer b21705b13fc94aed9b598a38da54d6ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:18:58.048688 31409 consensus_queue.cc:1035] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:18:58.049057 31405 consensus_queue.cc:1035] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:18:58.076789 30609 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:18:58.077070 30609 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:18:58.080932 30609 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Sending DeleteTablet for 3 replicas of tablet 8c4eac8793cf4037bb9abe42faad9511
I20260811 12:18:58.082278 31038 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4eac8793cf4037bb9abe42faad9511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:58 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:48208
I20260811 12:18:58.082396 31260 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4eac8793cf4037bb9abe42faad9511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:58 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:43306
I20260811 12:18:58.082477 30814 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4eac8793cf4037bb9abe42faad9511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:18:58 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:48458
I20260811 12:18:58.083240 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 30671
I20260811 12:18:58.094995 31435 tablet_replica.cc:330] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: stopping tablet replica
I20260811 12:18:58.095204 31437 tablet_replica.cc:330] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: stopping tablet replica
I20260811 12:18:58.095590 31437 raft_consensus.cc:2229] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:18:58.095563 31435 raft_consensus.cc:2229] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:18:58.096146 31437 raft_consensus.cc:2258] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:58.096364 31435 raft_consensus.cc:2258] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:58.097208 31438 tablet_replica.cc:330] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: stopping tablet replica
I20260811 12:18:58.097676 31438 raft_consensus.cc:2229] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:18:58.098213 31438 raft_consensus.cc:2258] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:18:58.098989 31437 ts_tablet_manager.cc:1905] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:58.100353 31435 ts_tablet_manager.cc:1905] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:58.105209 31438 ts_tablet_manager.cc:1905] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:18:58.106209 31437 ts_tablet_manager.cc:1918] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:18:58.106389 31437 log.cc:1196] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/8c4eac8793cf4037bb9abe42faad9511
I20260811 12:18:58.106948 31437 ts_tablet_manager.cc:1939] T 8c4eac8793cf4037bb9abe42faad9511 P 64bdd35fb68746bb83913ec4f840f35d: Deleting consensus metadata
I20260811 12:18:58.108511 31435 ts_tablet_manager.cc:1918] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:18:58.108696 31435 log.cc:1196] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/8c4eac8793cf4037bb9abe42faad9511
I20260811 12:18:58.108969 30591 catalog_manager.cc:4841] TS 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901): tablet 8c4eac8793cf4037bb9abe42faad9511 (table test-table [id=60771b2268c145df8df506568fb03059]) successfully deleted
I20260811 12:18:58.109114 31435 ts_tablet_manager.cc:1939] T 8c4eac8793cf4037bb9abe42faad9511 P b21705b13fc94aed9b598a38da54d6ba: Deleting consensus metadata
I20260811 12:18:58.110045 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.084137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.7.39.1:39543)
0811 12:18:58.084795 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:58.084801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:58.084820 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:18:58.084879 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:58.084884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:58.084961 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:58.085061 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:58.087290 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:58.088557 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:58.091001 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:58.091731 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:58.091866 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:58.096831 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:58.096857 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:58.096880 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:58.096929 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:58.102224 (+ 5295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:58.103300 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:58.103310 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:58.103314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:58.103518 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:58.105852 (+ 2334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:58.105858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:58.105862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:58.106593 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:58.106925 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:58.108363 (+ 1438us) server_negotiation.cc:299] Negotiation successful
0811 12:18:58.109668 (+ 1305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526}
I20260811 12:18:58.111096 30591 catalog_manager.cc:4841] TS b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745): tablet 8c4eac8793cf4037bb9abe42faad9511 (table test-table [id=60771b2268c145df8df506568fb03059]) successfully deleted
I20260811 12:18:58.111245 30914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:58.234576 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 30671
W20260811 12:18:58.270778 30591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.39.1:39543 (error 108)
I20260811 12:18:58.271121 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 30937
I20260811 12:18:58.273036 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.272112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48480)
0811 12:18:58.272670 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:58.272801 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:39543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":307,"threads_started":1}
W20260811 12:18:58.273343 30591 catalog_manager.cc:4584] TS 3d013cbbb9ad4920aadaebba30065933 (127.7.39.1:39543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c4eac8793cf4037bb9abe42faad9511: Network error: Client connection negotiation failed: client connection to 127.7.39.1:39543: connect: Connection refused (error 111)
I20260811 12:18:58.287830 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.272122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.7.39.2:44745)
0811 12:18:58.272322 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:58.272329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:58.272657 (+ 328us) server_negotiation.cc:407] Connection header received
0811 12:18:58.272735 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:58.272740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:58.272823 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:58.272935 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:58.274800 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:58.275689 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:58.277662 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:58.278096 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:58.278197 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:58.281593 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:58.281607 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:58.281625 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:58.281662 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:58.284737 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:58.285665 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:58.285672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:58.285675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:58.285823 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:58.286223 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:58.286226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:58.286228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:58.286623 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:58.286888 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:58.287214 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 12:18:58.287427 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:18:58.288632 31133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:58.327351 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.326735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48490)
0811 12:18:58.327007 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:58.327162 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:39543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 12:18:58.328605 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.328159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48504)
0811 12:18:58.328318 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:58.328411 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:39543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 12:18:58.389077 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.388574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48508)
0811 12:18:58.388806 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:58.388909 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:39543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 12:18:58.390311 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.389890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48518)
0811 12:18:58.390056 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:58.390147 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:39543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 12:18:58.407183 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 30937
I20260811 12:18:58.434998 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 31148
I20260811 12:18:58.449098 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.435776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.7.39.3:33901)
0811 12:18:58.435968 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:18:58.435974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:18:58.435995 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:18:58.436151 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:18:58.436155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:18:58.436235 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:18:58.436319 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:18:58.437763 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:58.438484 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:58.439938 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:18:58.440392 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:18:58.440536 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:18:58.443222 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:18:58.443238 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:18:58.443256 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:18:58.443299 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:18:58.446033 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:58.446842 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:58.446851 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:58.446853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:58.447006 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:18:58.447451 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:18:58.447455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:18:58.447457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:18:58.447881 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:18:58.448163 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 12:18:58.448529 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 12:18:58.448762 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:18:58.449855 31356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:58.471271 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.470727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48524)
0811 12:18:58.470986 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:58.471091 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:39543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 12:18:58.472431 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:58.471976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39543 (local address 127.0.0.1:48534)
0811 12:18:58.472207 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:58.472284 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:39543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 12:18:58.541350 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 31148
I20260811 12:18:58.565392 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 30558
I20260811 12:18:58.566722 30657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:18:58.681957 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 30558
I20260811 12:18:58.711871 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34191
--webserver_interface=127.7.39.62
--webserver_port=34439
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:18:59.062069 31448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:59.062403 31448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:59.062546 31448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:59.069972 31448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:18:59.070099 31448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:59.070184 31448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:59.070262 31448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:18:59.070324 31448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:18:59.080655 31448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=34439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:59.082120 31448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:59.083508 31448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:59.095362 31456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:59.095391 31458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:59.095439 31461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:59.095952 31448 server_base.cc:1034] running on GCE node
I20260811 12:18:59.096910 31448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:59.098346 31448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:59.099550 31448 hybrid_clock.cc:648] HybridClock initialized: now 1786450739099486 us; error 50 us; skew 500 ppm
I20260811 12:18:59.099876 31448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:59 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:59.103012 31448 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:18:59.104521 31448 webserver.cc:460] Webserver started at http://127.7.39.62:34439/ using document root <none> and password file <none>
I20260811 12:18:59.105283 31448 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:59.105406 31448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:59.111238 31448 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 12:18:59.114297 31469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:59.115741 31448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:18:59.115917 31448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "92995140ec2b42e6af70790eb20f5953"
format_stamp: "Formatted at 2026-08-11 12:18:52 on dist-test-slave-rmhg"
server_key: "6b9ed4016c1b98dc33235447974ea044"
server_key_iv: "60b704c762d50778b2c558f65137d7a9"
server_key_version: "encryptionkey@0"
I20260811 12:18:59.116928 31448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:59.140002 31448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:59.144348 31448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:59.144635 31448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:59.162551 31448 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:34191
I20260811 12:18:59.162575 31554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:34191 every 8 connection(s)
I20260811 12:18:59.164095 31448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:18:59.170807 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 31448
I20260811 12:18:59.171609 31555 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:18:59.171890 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:39543
--local_ip_for_outbound_sockets=127.7.39.1
--tserver_master_addrs=127.7.39.62:34191
--webserver_port=39535
--webserver_interface=127.7.39.1
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:18:59.176287 31555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Bootstrap starting.
I20260811 12:18:59.198861 31555 log.cc:824] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:59.215682 31555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:18:59.216243 31555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953: Bootstrap complete.
I20260811 12:18:59.225939 31555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:59.226862 31555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92995140ec2b42e6af70790eb20f5953, State: Initialized, Role: FOLLOWER
I20260811 12:18:59.227468 31555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:59.227674 31555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:18:59.227792 31555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:18:59.227952 31555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:18:59.231851 31555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:59.232368 31555 leader_election.cc:302] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92995140ec2b42e6af70790eb20f5953; no voters:
I20260811 12:18:59.232991 31555 leader_election.cc:288] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 12:18:59.233139 31560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:18:59.234221 31560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [term 3 LEADER]: Becoming Leader. State: Replica: 92995140ec2b42e6af70790eb20f5953, State: Running, Role: LEADER
I20260811 12:18:59.234786 31560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } }
I20260811 12:18:59.235260 31555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:18:59.237650 31566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92995140ec2b42e6af70790eb20f5953. Latest consensus state: current_term: 3 leader_uuid: "92995140ec2b42e6af70790eb20f5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } } }
I20260811 12:18:59.237680 31562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "92995140ec2b42e6af70790eb20f5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92995140ec2b42e6af70790eb20f5953" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34191 } } }
I20260811 12:18:59.238117 31562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:59.238122 31566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92995140ec2b42e6af70790eb20f5953 [sys.catalog]: This master's current role is: LEADER
I20260811 12:18:59.251968 31573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:18:59.261802 31573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:18:59.263368 31573 catalog_manager.cc:1259] Loaded cluster ID: 860831a079694580aa8a3fab1a01286d
I20260811 12:18:59.263499 31573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 12:18:59.265982 31573 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9b01b699ad google::LogMessage::Fail() at ??:0
@ 0x7f9b01b6c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f9b01b694de google::LogMessage::Flush() at ??:0
@ 0x7f9b01b6aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9b175a0512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f9b1759d976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9b17625610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f9b17625439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9b047b5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9b047b891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f9b047b8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9b04793eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9b07a6d6db start_thread at ??:0
@ 0x7f9b00a0361f clone at ??:0
W20260811 12:18:59.589710 31557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:18:59.590029 31557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:18:59.590161 31557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:18:59.597678 31557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:18:59.597801 31557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:18:59.597958 31557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:18:59.608067 31557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:39543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=39535
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:18:59.609512 31557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:18:59.610922 31557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:18:59.625116 31623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:59.625174 31629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:18:59.625741 31625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:18:59.626205 31557 server_base.cc:1034] running on GCE node
I20260811 12:18:59.626883 31557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:18:59.627942 31557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:18:59.629180 31557 hybrid_clock.cc:648] HybridClock initialized: now 1786450739629084 us; error 62 us; skew 500 ppm
I20260811 12:18:59.629523 31557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:18:59 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:18:59.632794 31557 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:18:59.634359 31557 webserver.cc:460] Webserver started at http://127.7.39.1:39535/ using document root <none> and password file <none>
I20260811 12:18:59.635102 31557 fs_manager.cc:359] Metadata directory not provided
I20260811 12:18:59.635226 31557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:18:59.641211 31557 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 12:18:59.644366 31639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:18:59.645947 31557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:18:59.646142 31557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "3d013cbbb9ad4920aadaebba30065933"
format_stamp: "Formatted at 2026-08-11 12:18:53 on dist-test-slave-rmhg"
server_key: "c6a7388778ecc2aaba598d55d4c607f3"
server_key_iv: "026c86de5a793cb58599930e00a2c0e1"
server_key_version: "encryptionkey@0"
I20260811 12:18:59.647157 31557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:18:59.664736 31557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:18:59.668987 31557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:18:59.669318 31557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:18:59.670392 31557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:18:59.672230 31649 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 12:18:59.678246 31557 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 12:18:59.678382 31557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 12:18:59.678511 31557 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 12:18:59.681825 31557 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 12:18:59.681902 31557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260811 12:18:59.682250 31649 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Bootstrap starting.
I20260811 12:18:59.691095 31649 log.cc:824] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Log is configured to *not* fsync() on all Append() calls
I20260811 12:18:59.706769 31649 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:18:59.707628 31649 tablet_bootstrap.cc:492] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Bootstrap complete.
I20260811 12:18:59.708266 31649 ts_tablet_manager.cc:1397] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Time spent bootstrapping tablet: real 0.026s user 0.021s sys 0.001s
I20260811 12:18:59.718312 31557 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:39543
I20260811 12:18:59.718355 31817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:39543 every 8 connection(s)
I20260811 12:18:59.717926 31649 raft_consensus.cc:348] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:59.719076 31649 raft_consensus.cc:729] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d013cbbb9ad4920aadaebba30065933, State: Initialized, Role: FOLLOWER
I20260811 12:18:59.719738 31557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:18:59.719802 31649 consensus_queue.cc:260] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:18:59.721788 31649 ts_tablet_manager.cc:1428] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 12:18:59.724720 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:59.722793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:48689)
0811 12:18:59.723604 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:59.723808 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":360,"threads_started":1}
W20260811 12:18:59.725461 31818 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:34191 (0 consecutive failures): Network error: Failed to ping master at 127.7.39.62:34191: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
I20260811 12:18:59.726312 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:59.725977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:34275)
0811 12:18:59.726123 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:59.726200 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:18:59.727133 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:18:59.726821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:43575)
0811 12:18:59.726942 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 12:18:59.727015 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 12:18:59.727373 31818 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:18:59.729839 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 31557
I20260811 12:18:59.730458 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:44745
--local_ip_for_outbound_sockets=127.7.39.2
--tserver_master_addrs=127.7.39.62:34191
--webserver_port=35601
--webserver_interface=127.7.39.2
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:19:00.072923 31824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:00.073288 31824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:00.073460 31824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:00.081043 31824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:00.081197 31824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:00.081336 31824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:19:00.091305 31824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:44745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=35601
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:00.092653 31824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:00.094026 31824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:00.109675 31830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:00.109596 31831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:00.109697 31836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:00.110795 31824 server_base.cc:1034] running on GCE node
I20260811 12:19:00.111568 31824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:00.112674 31824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:00.113940 31824 hybrid_clock.cc:648] HybridClock initialized: now 1786450740113868 us; error 63 us; skew 500 ppm
I20260811 12:19:00.114315 31824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:00 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:00.117784 31824 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:19:00.119467 31824 webserver.cc:460] Webserver started at http://127.7.39.2:35601/ using document root <none> and password file <none>
I20260811 12:19:00.120198 31824 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:00.120312 31824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:00.126164 31824 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:19:00.129380 31844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:00.130941 31824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:19:00.131135 31824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "b21705b13fc94aed9b598a38da54d6ba"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "43bfbbd88f831912b87a3755985e477b"
server_key_iv: "cd733863f780e11a35d60824342ac0e6"
server_key_version: "encryptionkey@0"
I20260811 12:19:00.132200 31824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:00.161123 31824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:00.165928 31824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:00.166224 31824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:00.167331 31824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:19:00.168932 31824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:19:00.169041 31824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:00.169205 31824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:19:00.169287 31824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:00.201676 31824 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:44745
I20260811 12:19:00.201763 32021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:44745 every 8 connection(s)
I20260811 12:19:00.203209 31824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:19:00.208242 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:00.206255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:48583)
0811 12:19:00.207058 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:00.207294 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":314,"threads_started":1}
W20260811 12:19:00.209043 32022 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:34191 (0 consecutive failures): Network error: Failed to ping master at 127.7.39.62:34191: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
I20260811 12:19:00.209528 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 31824
I20260811 12:19:00.210114 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:00.209699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:46217)
0811 12:19:00.209873 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:00.209967 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 12:19:00.210246 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:33901
--local_ip_for_outbound_sockets=127.7.39.3
--tserver_master_addrs=127.7.39.62:34191
--webserver_port=38963
--webserver_interface=127.7.39.3
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:45203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:19:00.211088 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:00.210718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:33687)
0811 12:19:00.210861 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:00.210952 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:19:00.211325 32022 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:19:00.565069 32027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:00.565451 32027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:00.565608 32027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:00.573431 32027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:00.573563 32027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:00.573732 32027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:19:00.584148 32027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:45203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:33901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=38963
--tserver_master_addrs=127.7.39.62:34191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:00.585670 32027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:00.587102 32027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:00.601238 32041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:00.601266 32044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:00.601207 32040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:00.601998 32027 server_base.cc:1034] running on GCE node
I20260811 12:19:00.602689 32027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:00.603703 32027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:00.604943 32027 hybrid_clock.cc:648] HybridClock initialized: now 1786450740604880 us; error 41 us; skew 500 ppm
I20260811 12:19:00.605330 32027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:00 dist-test-slave-rmhg krb5kdc[29642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:00.608650 32027 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:19:00.610349 32027 webserver.cc:460] Webserver started at http://127.7.39.3:38963/ using document root <none> and password file <none>
I20260811 12:19:00.611070 32027 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:00.611177 32027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:00.616796 32027 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:19:00.619745 32053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:00.621367 32027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:19:00.621521 32027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "64bdd35fb68746bb83913ec4f840f35d"
format_stamp: "Formatted at 2026-08-11 12:18:54 on dist-test-slave-rmhg"
server_key: "fdf69fb0e8aa44886ee820bd4ef7f2d1"
server_key_iv: "11fccc9fe940e3493d570233580be3f7"
server_key_version: "encryptionkey@0"
I20260811 12:19:00.622561 32027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:00.650543 32027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:00.654863 32027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:00.655112 32027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:00.656095 32027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:19:00.657474 32027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:19:00.657562 32027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:00.657652 32027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:19:00.657707 32027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:00.691694 32027 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:33901
I20260811 12:19:00.691781 32238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:33901 every 8 connection(s)
I20260811 12:19:00.693274 32027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:19:00.698467 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:00.696362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:59175)
0811 12:19:00.697246 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:00.697500 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":336,"threads_started":1}
W20260811 12:19:00.699265 32240 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:34191 (0 consecutive failures): Network error: Failed to ping master at 127.7.39.62:34191: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
I20260811 12:19:00.700268 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:00.699868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:48587)
0811 12:19:00.700053 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:00.700144 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 12:19:00.700290 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 32027
I20260811 12:19:00.701308 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:00.700880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:43017)
0811 12:19:00.701029 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:00.701131 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:19:00.701675 32240 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:19:00.728670 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:00.727906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:36633)
0811 12:19:00.728349 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:00.728439 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":200,"threads_started":1}
I20260811 12:19:01.213135 32248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:01.212121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:38969)
0811 12:19:01.212726 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:01.212825 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":278,"threads_started":1}
I20260811 12:19:01.467396 32250 raft_consensus.cc:482] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:19:01.467710 32250 raft_consensus.cc:504] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:19:01.469410 32250 leader_election.cc:288] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745), 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901)
Aug 11 12:19:01 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
Aug 11 12:19:01 dist-test-slave-rmhg krb5kdc[29642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:19:01.502035 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:01.469993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:35403 (local address 127.7.39.2:44745)
0811 12:19:01.472000 (+ 2007us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:01.472016 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:01.472063 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:19:01.474345 (+ 2282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:01.474358 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:01.474890 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:01.475332 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:01.478240 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:01.480172 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:01.483045 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:01.484102 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:01.484300 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:01.493988 (+ 9688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:01.494004 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:01.494027 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:01.494123 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:01.498068 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:01.499022 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:01.499030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:01.499036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:01.499211 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:01.499688 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:01.499692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:01.499694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:01.500251 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:01.500430 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:01.501269 (+ 839us) server_negotiation.cc:299] Negotiation successful
0811 12:19:01.501636 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1800,"thread_start_us":1623,"threads_started":1}
I20260811 12:19:01.502043 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:01.470721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:56037 (local address 127.7.39.3:33901)
0811 12:19:01.472265 (+ 1544us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:01.472279 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:01.472315 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:19:01.474345 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:01.474358 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:01.474890 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:01.475302 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:01.478240 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:01.480162 (+ 1922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:01.483045 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:01.484103 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:01.484300 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:01.494090 (+ 9790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:01.494103 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:01.494116 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:01.494162 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:01.498057 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:01.499030 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:01.499038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:01.499044 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:01.499228 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:01.499689 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:01.499693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:01.499695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:01.500246 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:01.500429 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:01.501280 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 12:19:01.501654 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1374,"thread_start_us":1229,"threads_started":1}
I20260811 12:19:01.502046 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:01.469667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:44745 (local address 127.7.39.1:35403)
0811 12:19:01.470184 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:01.470226 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:01.470875 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:01.475514 (+ 4639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:01.475527 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:01.476729 (+ 1202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:19:01.477988 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:01.478021 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:01.480405 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:01.480412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:01.482872 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:01.482892 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:01.484374 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:01.484380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:01.484814 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:01.490032 (+ 5218us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:19:01.490087 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:19:01.493512 (+ 3425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:19:01.498251 (+ 4739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:01.498260 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:01.498274 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:01.498856 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:01.499328 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:01.499334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:01.499338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:01.499564 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:01.500391 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:19:01.500400 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:19:01.500722 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 12:19:01.500972 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 12:19:01.501556 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":308,"spinlock_wait_cycles":60672,"thread_start_us":245,"threads_started":1}
I20260811 12:19:01.502128 32256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:01.470351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33901 (local address 127.7.39.1:56037)
0811 12:19:01.471656 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:01.471674 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:01.471787 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:01.475514 (+ 3727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:01.475527 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:01.476729 (+ 1202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:19:01.477988 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:01.478021 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:01.480417 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:01.480421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:01.482872 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:01.482892 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:01.484353 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:01.484362 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:01.484809 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:01.490032 (+ 5223us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:19:01.490086 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:19:01.493732 (+ 3646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:19:01.498251 (+ 4519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:01.498260 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:01.498274 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:01.498868 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:01.499344 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:01.499348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:01.499350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:01.499565 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:01.500391 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:19:01.500400 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:19:01.500722 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 12:19:01.500972 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 12:19:01.501737 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1158,"mutex_wait_us":582,"spinlock_wait_cycles":37504,"thread_start_us":1000,"threads_started":1}
W20260811 12:19:01.505920 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
W20260811 12:19:01.506243 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
I20260811 12:19:01.506351 31645 leader_election.cc:302] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d013cbbb9ad4920aadaebba30065933; no voters: 64bdd35fb68746bb83913ec4f840f35d, b21705b13fc94aed9b598a38da54d6ba
I20260811 12:19:01.506735 32250 raft_consensus.cc:2726] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:19:01.703051 32261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:01.702162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:60211)
0811 12:19:01.702652 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:01.702799 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":233,"threads_started":1}
I20260811 12:19:01.730171 32256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:01.729680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:37393)
0811 12:19:01.729879 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:01.729999 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 12:19:02.214823 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:02.214111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:32957)
0811 12:19:02.214558 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:02.214651 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
I20260811 12:19:02.704929 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:02.704116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:57025)
0811 12:19:02.704624 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:02.704757 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":247,"threads_started":1}
I20260811 12:19:02.731812 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:02.731069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:32923)
0811 12:19:02.731542 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:02.731626 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":238,"threads_started":1}
I20260811 12:19:03.137977 32268 raft_consensus.cc:482] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:19:03.138280 32268 raft_consensus.cc:504] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:19:03.138983 32268 leader_election.cc:288] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745), 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901)
W20260811 12:19:03.140000 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
W20260811 12:19:03.140236 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
I20260811 12:19:03.140388 31645 leader_election.cc:302] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d013cbbb9ad4920aadaebba30065933; no voters: 64bdd35fb68746bb83913ec4f840f35d, b21705b13fc94aed9b598a38da54d6ba
I20260811 12:19:03.140642 32268 raft_consensus.cc:2726] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:19:03.216702 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:03.215862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:48571)
0811 12:19:03.216411 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:03.216508 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":280,"threads_started":1}
I20260811 12:19:03.706758 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:03.705939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:59793)
0811 12:19:03.706464 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:03.706581 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
I20260811 12:19:03.733592 32273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:03.732760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:51493)
0811 12:19:03.733310 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:03.733414 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":273,"threads_started":1}
I20260811 12:19:04.218480 32274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:04.217700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:49169)
0811 12:19:04.218190 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:04.218284 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":231,"threads_started":1}
I20260811 12:19:04.708999 32275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:04.707881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:60225)
0811 12:19:04.708605 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:04.708750 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":419,"threads_started":1}
I20260811 12:19:04.735468 32276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:04.734628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:34053)
0811 12:19:04.735190 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:04.735277 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":250,"threads_started":1}
I20260811 12:19:05.220289 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:05.219494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:36741)
0811 12:19:05.220013 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:05.220104 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":286,"threads_started":1}
I20260811 12:19:05.710628 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:05.709915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:42519)
0811 12:19:05.710355 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:05.710439 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":232,"threads_started":1}
I20260811 12:19:05.737337 32279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:05.736390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:39159)
0811 12:19:05.736939 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:05.737066 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":301,"threads_started":1}
I20260811 12:19:06.222103 32280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:06.221263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:42919)
0811 12:19:06.221785 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:06.221878 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":271,"threads_started":1}
I20260811 12:19:06.434216 32284 raft_consensus.cc:482] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:19:06.434454 32284 raft_consensus.cc:504] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:19:06.435098 32284 leader_election.cc:288] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745), 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901)
W20260811 12:19:06.436204 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
W20260811 12:19:06.436436 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
I20260811 12:19:06.436558 31645 leader_election.cc:302] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d013cbbb9ad4920aadaebba30065933; no voters: 64bdd35fb68746bb83913ec4f840f35d, b21705b13fc94aed9b598a38da54d6ba
I20260811 12:19:06.436836 32284 raft_consensus.cc:2726] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:19:06.712420 32287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:06.711556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:39463)
0811 12:19:06.712109 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:06.712209 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":284,"threads_started":1}
I20260811 12:19:06.739283 32293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:06.738460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:49729)
0811 12:19:06.738945 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:06.739050 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":241,"threads_started":1}
I20260811 12:19:07.224031 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:07.223118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:50563)
0811 12:19:07.223730 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:07.223814 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":305,"threads_started":1}
I20260811 12:19:07.714293 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:07.713426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:33795)
0811 12:19:07.713986 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:07.714099 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":240,"threads_started":1}
I20260811 12:19:07.741020 32297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:07.740230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:35311)
0811 12:19:07.740752 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:07.740846 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":274,"threads_started":1}
I20260811 12:19:08.225942 32299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:08.225077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:52407)
0811 12:19:08.225592 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:08.225688 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":256,"threads_started":1}
I20260811 12:19:08.715996 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:08.715236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:50307)
0811 12:19:08.715715 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:08.715804 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":219,"threads_started":1}
I20260811 12:19:08.742807 32303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:08.742033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:52797)
0811 12:19:08.742528 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:08.742609 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":240,"threads_started":1}
I20260811 12:19:09.227912 32305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:09.227029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:57071)
0811 12:19:09.227575 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:09.227729 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":243,"threads_started":1}
I20260811 12:19:09.717701 32306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:09.716952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:57921)
0811 12:19:09.717442 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:09.717536 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":262,"threads_started":1}
I20260811 12:19:09.744565 32307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:09.743786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:60555)
0811 12:19:09.744287 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:09.744377 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":229,"threads_started":1}
I20260811 12:19:10.229905 32308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:10.229055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:55295)
0811 12:19:10.229606 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:10.229703 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":308,"threads_started":1}
I20260811 12:19:10.389386 32309 raft_consensus.cc:482] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:19:10.389601 32309 raft_consensus.cc:504] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:19:10.390225 32309 leader_election.cc:288] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745), 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901)
W20260811 12:19:10.391180 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
W20260811 12:19:10.391417 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
I20260811 12:19:10.391513 31645 leader_election.cc:302] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d013cbbb9ad4920aadaebba30065933; no voters: 64bdd35fb68746bb83913ec4f840f35d, b21705b13fc94aed9b598a38da54d6ba
I20260811 12:19:10.391763 32309 raft_consensus.cc:2726] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:19:10.719645 32311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:10.718706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:51175)
0811 12:19:10.719296 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:10.719415 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":327,"threads_started":1}
I20260811 12:19:10.746263 32312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:10.745437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:38231)
0811 12:19:10.745961 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:10.746059 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":252,"threads_started":1}
I20260811 12:19:11.231683 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:11.230918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:54541)
0811 12:19:11.231417 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:11.231505 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":230,"threads_started":1}
I20260811 12:19:11.721586 32314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:11.720714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:51309)
0811 12:19:11.721329 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:11.721424 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":278,"threads_started":1}
I20260811 12:19:11.748167 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:11.747301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:50673)
0811 12:19:11.747866 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:11.747965 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":297,"threads_started":1}
I20260811 12:19:12.233525 32318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:12.232728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:36765)
0811 12:19:12.233249 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:12.233343 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":236,"threads_started":1}
I20260811 12:19:12.723343 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:12.722614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:40517)
0811 12:19:12.723078 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:12.723170 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":225,"threads_started":1}
I20260811 12:19:12.749861 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:12.749081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:59639)
0811 12:19:12.749590 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:12.749673 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":291,"threads_started":1}
I20260811 12:19:13.235221 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:13.234470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:60147)
0811 12:19:13.234945 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:13.235052 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":231,"threads_started":1}
I20260811 12:19:13.725023 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:13.724298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:58133)
0811 12:19:13.724757 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:13.724853 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":223,"threads_started":1}
I20260811 12:19:13.751518 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:13.750793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:33237)
0811 12:19:13.751234 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:13.751326 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":218,"threads_started":1}
I20260811 12:19:14.236980 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:14.236153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:33767)
0811 12:19:14.236671 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:14.236759 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":279,"threads_started":1}
I20260811 12:19:14.726706 32326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:14.725953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:41331)
0811 12:19:14.726398 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:14.726481 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":236,"threads_started":1}
I20260811 12:19:14.753526 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:14.752733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:36957)
0811 12:19:14.753247 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:14.753334 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":240,"threads_started":1}
I20260811 12:19:15.177670 32330 raft_consensus.cc:482] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:19:15.177906 32330 raft_consensus.cc:504] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21705b13fc94aed9b598a38da54d6ba" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44745 } } peers { permanent_uuid: "64bdd35fb68746bb83913ec4f840f35d" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33901 } } peers { permanent_uuid: "3d013cbbb9ad4920aadaebba30065933" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39543 } }
I20260811 12:19:15.178668 32330 leader_election.cc:288] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745), 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901)
W20260811 12:19:15.179771 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b21705b13fc94aed9b598a38da54d6ba (127.7.39.2:44745): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
W20260811 12:19:15.179984 31645 leader_election.cc:341] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64bdd35fb68746bb83913ec4f840f35d (127.7.39.3:33901): Not found: Tablet not found: 8c4eac8793cf4037bb9abe42faad9511
I20260811 12:19:15.180094 31645 leader_election.cc:302] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d013cbbb9ad4920aadaebba30065933; no voters: 64bdd35fb68746bb83913ec4f840f35d, b21705b13fc94aed9b598a38da54d6ba
I20260811 12:19:15.180375 32330 raft_consensus.cc:2726] T 8c4eac8793cf4037bb9abe42faad9511 P 3d013cbbb9ad4920aadaebba30065933 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:19:15.238894 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:15.238136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.2:58433)
0811 12:19:15.238615 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:15.238730 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":229,"threads_started":1}
I20260811 12:19:15.706765 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 31557
I20260811 12:19:15.728193 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:15.707955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.7.39.1:39543)
0811 12:19:15.708743 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:15.708751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:15.709207 (+ 456us) server_negotiation.cc:407] Connection header received
0811 12:19:15.709468 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:15.709478 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:15.709780 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:15.709971 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:15.711324 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:15.712306 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:15.713674 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:15.714251 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:15.714348 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:15.721487 (+ 7139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:15.721506 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:15.721528 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:15.721576 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:15.725001 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:15.725827 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:15.725835 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:15.725837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:15.725990 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:15.726466 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:15.726471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:15.726473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:15.726912 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:15.727200 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:15.727589 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 12:19:15.727841 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":532,"threads_started":1}
I20260811 12:19:15.728667 32334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:15.727900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.3:45571)
0811 12:19:15.728390 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:15.728488 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":252,"threads_started":1}
I20260811 12:19:15.729609 31812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:15.819067 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 31557
I20260811 12:19:15.819118 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:15.818104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:34191 (local address 127.7.39.1:47063)
0811 12:19:15.818661 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:15.818808 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:34191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":274,"threads_started":1}
I20260811 12:19:15.844209 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 31824
I20260811 12:19:15.857566 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:15.845014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.7.39.2:44745)
0811 12:19:15.845530 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:15.845534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:15.845551 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:19:15.845607 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:15.845611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:15.845679 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:15.845768 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:15.847150 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:15.847963 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:15.849233 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:15.849698 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:15.849791 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:15.852342 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:15.852347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:15.852350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:15.852383 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:15.854954 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:15.855613 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:15.855617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:15.855619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:15.855758 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:15.856097 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:15.856101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:15.856103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:15.856382 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:15.856596 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:15.856946 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 12:19:15.857130 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":297,"threads_started":1}
I20260811 12:19:15.858346 32016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:15.942044 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 31824
I20260811 12:19:15.966850 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 32027
I20260811 12:19:15.980928 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:15.967686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.7.39.3:33901)
0811 12:19:15.968311 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:15.968316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:15.968333 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:19:15.968386 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:15.968390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:15.968461 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:15.968561 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:15.970034 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:15.970817 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:15.972210 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:15.972661 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:15.972779 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:15.975421 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:15.975427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:15.975430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:15.975464 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:15.978213 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:15.978940 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:15.978945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:15.978947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:15.979090 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:15.979472 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:15.979477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:15.979479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:15.979757 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:15.980035 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:15.980312 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 12:19:15.980542 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":349,"threads_started":1}
I20260811 12:19:15.981715 32231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:16.071610 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 32027
2026-08-11T12:19:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25116 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:19:16 dist-test-slave-rmhg krb5kdc[32347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:19:16 dist-test-slave-rmhg krb5kdc[32347](info): set up 2 sockets
Aug 11 12:19:16 dist-test-slave-rmhg krb5kdc[32347](info): commencing operation
krb5kdc: starting...
W20260811 12:19:17.333220 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:19:17 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:19:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:19:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:19:17.640308 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40833
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:40833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:19:17.991920 32369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:17.992237 32369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:17.992357 32369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:17.999718 32369 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:19:17.999840 32369 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:19:17.999886 32369 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:19:17.999958 32369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:19:18.000049 32369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:18.000130 32369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:18.000172 32369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:19:18.000250 32369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:19:18.010644 32369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:40833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:18.012012 32369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:18.013521 32369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:18.025826 32384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:18.025861 32380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:18.025796 32379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:18.026607 32369 server_base.cc:1034] running on GCE node
I20260811 12:19:18.027364 32369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:18.028820 32369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:18.030058 32369 hybrid_clock.cc:648] HybridClock initialized: now 1786450758029989 us; error 30 us; skew 500 ppm
I20260811 12:19:18.030467 32369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:18 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450758, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:18.033812 32369 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:19:18.035355 32369 webserver.cc:460] Webserver started at http://127.7.39.62:45785/ using document root <none> and password file <none>
I20260811 12:19:18.036080 32369 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:18.036190 32369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:18.036507 32369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:19:18.039690 32369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "3cb90125183c4b5a99b12499648bebda"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "3979d9da92e985efbf327800de55442f"
server_key_iv: "a740aa4b2b873f67c4df21426138cbf7"
server_key_version: "encryptionkey@0"
I20260811 12:19:18.040510 32369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "3cb90125183c4b5a99b12499648bebda"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "3979d9da92e985efbf327800de55442f"
server_key_iv: "a740aa4b2b873f67c4df21426138cbf7"
server_key_version: "encryptionkey@0"
I20260811 12:19:18.045969 32369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 12:19:18.049597 32397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:18.051033 32369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 12:19:18.051221 32369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "3cb90125183c4b5a99b12499648bebda"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "3979d9da92e985efbf327800de55442f"
server_key_iv: "a740aa4b2b873f67c4df21426138cbf7"
server_key_version: "encryptionkey@0"
I20260811 12:19:18.051435 32369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:18.065971 32369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:18.070061 32369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:18.070323 32369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:18.089068 32369 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:40833
I20260811 12:19:18.089093 32478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:40833 every 8 connection(s)
I20260811 12:19:18.090610 32369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:19:18.094656 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:19:18.100896 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 32369
I20260811 12:19:18.101145 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:19:18.101887 7324 external_mini_cluster.cc:1428] Setting key 1353f3f0b8c3afc59518522af47f6e05
I20260811 12:19:18.106093 32481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Bootstrap starting.
I20260811 12:19:18.110008 32481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Neither blocks nor log segments found. Creating new log.
I20260811 12:19:18.111151 32481 log.cc:824] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Log is configured to *not* fsync() on all Append() calls
I20260811 12:19:18.114485 32481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: No bootstrap required, opened a new log
Aug 11 12:19:18 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:19:18.123622 32481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:19:18.124037 32481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:19:18.124136 32481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb90125183c4b5a99b12499648bebda, State: Initialized, Role: FOLLOWER
I20260811 12:19:18.124778 32481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:19:18.125051 32481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:19:18.125207 32481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:19:18.125416 32481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:19:18.127485 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:18.104153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.7.39.62:40833)
0811 12:19:18.104994 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:18.105009 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:18.105055 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:19:18.106929 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:18.106956 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:18.107379 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:18.107760 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:18.109686 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:18.111133 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:18.112787 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:18.113623 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:18.113760 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:18.119012 (+ 5252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:18.119021 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:18.119040 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:18.119100 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:18.122745 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:18.123557 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:18.123565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:18.123572 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:18.123796 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:18.124293 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:18.124297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:18.124299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:18.125015 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:18.125328 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:18.125847 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 12:19:18.126242 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":394,"threads_started":1}
I20260811 12:19:18.128124 32481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:19:18.128659 32481 leader_election.cc:302] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cb90125183c4b5a99b12499648bebda; no voters:
I20260811 12:19:18.129331 32481 leader_election.cc:288] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:19:18.129453 32489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:19:18.131732 32481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:19:18.133459 32489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 1 LEADER]: Becoming Leader. State: Replica: 3cb90125183c4b5a99b12499648bebda, State: Running, Role: LEADER
I20260811 12:19:18.134191 32489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:19:18.138348 32492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cb90125183c4b5a99b12499648bebda. Latest consensus state: current_term: 1 leader_uuid: "3cb90125183c4b5a99b12499648bebda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } } }
I20260811 12:19:18.138849 32491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cb90125183c4b5a99b12499648bebda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } } }
I20260811 12:19:18.139240 32491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: This master's current role is: LEADER
I20260811 12:19:18.139236 32492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: This master's current role is: LEADER
W20260811 12:19:18.143728 32514 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:19:18.143896 32514 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:19:18.145653 32515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:19:18.147373 32515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:19:18.155411 32515 catalog_manager.cc:1347] Generated new cluster ID: add29195478a495c8454f2ea4effcf90
I20260811 12:19:18.155540 32515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:19:18.211359 32515 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:19:18.212741 32515 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:19:18.227563 32515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Generated new TSK 0
I20260811 12:19:18.228761 32515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:19:18.309235 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40833
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:19:18.698537 32522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:18.698822 32522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:18.698925 32522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:18.706125 32522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:18.706240 32522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:18.706359 32522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:19:18.716883 32522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:18.718183 32522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:18.719473 32522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:18.732848 32532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:18.732775 32529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:18.732817 32528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:18.733563 32522 server_base.cc:1034] running on GCE node
I20260811 12:19:18.734161 32522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:18.735095 32522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:18.736312 32522 hybrid_clock.cc:648] HybridClock initialized: now 1786450758736247 us; error 53 us; skew 500 ppm
I20260811 12:19:18.736652 32522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:18 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450758, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:18.739847 32522 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:19:18.741425 32522 webserver.cc:460] Webserver started at http://127.7.39.1:41983/ using document root <none> and password file <none>
I20260811 12:19:18.742202 32522 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:18.742334 32522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:18.742664 32522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:19:18.745810 32522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "57b0929c4ce2433eaef580e795381680"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "f9a2ec521d7eb2c476b5d2209d604dbe"
server_key_iv: "42a7eb60dfbfbd3fbb164e6c481c8eb4"
server_key_version: "encryptionkey@0"
I20260811 12:19:18.746585 32522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "57b0929c4ce2433eaef580e795381680"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "f9a2ec521d7eb2c476b5d2209d604dbe"
server_key_iv: "42a7eb60dfbfbd3fbb164e6c481c8eb4"
server_key_version: "encryptionkey@0"
I20260811 12:19:18.751818 32522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:19:18.755429 32543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:18.756955 32522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:19:18.757120 32522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "57b0929c4ce2433eaef580e795381680"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "f9a2ec521d7eb2c476b5d2209d604dbe"
server_key_iv: "42a7eb60dfbfbd3fbb164e6c481c8eb4"
server_key_version: "encryptionkey@0"
I20260811 12:19:18.757396 32522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:18.778910 32522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:18.783633 32522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:18.783932 32522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:18.785001 32522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:19:18.786491 32522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:19:18.786576 32522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:18.786706 32522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:19:18.786763 32522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:18.819593 32522 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:38405
I20260811 12:19:18.819639 32725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:38405 every 8 connection(s)
I20260811 12:19:18.820909 32522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:19:18.827317 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 32522
I20260811 12:19:18.827524 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:19:18.828213 7324 external_mini_cluster.cc:1428] Setting key d388c678375498ee5c9ff80ab74a6794
Aug 11 12:19:18 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450758, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:19:18.843031 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:18.823697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:60685 (local address 127.7.39.62:40833)
0811 12:19:18.824167 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:18.824172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:18.825301 (+ 1129us) server_negotiation.cc:407] Connection header received
0811 12:19:18.827235 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:18.827241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:18.827328 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:18.827433 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:18.829268 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:18.830348 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:18.832684 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:18.833138 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:18.833289 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:18.837318 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:18.837324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:18.837327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:18.837360 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:18.839813 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:18.840558 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:18.840563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:18.840565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:18.840702 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:18.841123 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:18.841127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:18.841130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:18.841441 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:18.841581 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:18.842413 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 12:19:18.842663 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":254,"threads_started":1}
I20260811 12:19:18.844009 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:18.824289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:60685)
0811 12:19:18.825101 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:18.825185 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:18.826923 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:18.827771 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:18.827780 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:18.828369 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:19:18.829004 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:18.829033 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:18.830522 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:18.830528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:18.832505 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:18.832523 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:18.833421 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:18.833427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:18.833714 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:18.834444 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:19:18.834480 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:19:18.837060 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:19:18.839951 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:18.839961 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:18.839980 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:18.840430 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:18.840809 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:18.840815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:18.840818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:18.841009 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:18.841571 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:19:18.841579 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:19:18.842011 (+ 432us) client_negotiation.cc:769] Sending connection context
0811 12:19:18.842267 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 12:19:18.842837 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":357,"threads_started":1}
I20260811 12:19:18.845731 32726 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40833
I20260811 12:19:18.846019 32726 heartbeater.cc:464] Registering TS with master...
I20260811 12:19:18.846766 32726 heartbeater.cc:511] Master 127.7.39.62:40833 requested a full tablet report, sending...
I20260811 12:19:18.848816 32421 ts_manager.cc:194] Registered new tserver with Master: 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405)
I20260811 12:19:18.850286 32421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:60685
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:19:18.892868 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40833
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:19:19.240679 32744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:19.240995 32744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:19.241135 32744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:19.248637 32744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:19.248770 32744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:19.248924 32744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:19:19.259240 32744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:19.260649 32744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:19.262115 32744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:19.276048 32755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:19.276103 32756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:19.276082 32761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:19.276785 32744 server_base.cc:1034] running on GCE node
I20260811 12:19:19.277525 32744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:19.278556 32744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:19.279789 32744 hybrid_clock.cc:648] HybridClock initialized: now 1786450759279721 us; error 64 us; skew 500 ppm
I20260811 12:19:19.280138 32744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:19 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450759, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:19.283504 32744 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:19:19.285184 32744 webserver.cc:460] Webserver started at http://127.7.39.2:45583/ using document root <none> and password file <none>
I20260811 12:19:19.285926 32744 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:19.286055 32744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:19.286372 32744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:19:19.289580 32744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "aab77284fd3b4b65926810ae8e50e95e"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "4ff0b7575311fa65b83094b61acde690"
server_key_iv: "2310cc19f749d003a7dd70e7fda6ffac"
server_key_version: "encryptionkey@0"
I20260811 12:19:19.290408 32744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "aab77284fd3b4b65926810ae8e50e95e"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "4ff0b7575311fa65b83094b61acde690"
server_key_iv: "2310cc19f749d003a7dd70e7fda6ffac"
server_key_version: "encryptionkey@0"
I20260811 12:19:19.296314 32744 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 12:19:19.300452 301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:19.302218 32744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:19:19.302421 32744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "aab77284fd3b4b65926810ae8e50e95e"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "4ff0b7575311fa65b83094b61acde690"
server_key_iv: "2310cc19f749d003a7dd70e7fda6ffac"
server_key_version: "encryptionkey@0"
I20260811 12:19:19.302664 32744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:19.322763 32744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:19.327463 32744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:19.327819 32744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:19.328917 32744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:19:19.330524 32744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:19:19.330622 32744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:19.330755 32744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:19:19.330811 32744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:19.363109 32744 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:33135
I20260811 12:19:19.363183 485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:33135 every 8 connection(s)
I20260811 12:19:19.364419 32744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:19:19.374464 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 32744
I20260811 12:19:19.374663 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:19:19.375180 7324 external_mini_cluster.cc:1428] Setting key 65da9d7d793bd04f921abe9c30e7ccba
Aug 11 12:19:19 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450759, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:19:19.386310 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.367292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:56703 (local address 127.7.39.62:40833)
0811 12:19:19.367931 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:19.367937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:19.369132 (+ 1195us) server_negotiation.cc:407] Connection header received
0811 12:19:19.371114 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:19.371120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:19.371197 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:19.371303 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:19.373000 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.373858 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.375749 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.376359 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.376475 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:19.380327 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:19.380332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:19.380335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:19.380373 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:19.382950 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:19.383753 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:19.383759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:19.383761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:19.383897 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:19.384349 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:19.384353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:19.384355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:19.384686 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:19.384845 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:19.385657 (+ 812us) server_negotiation.cc:299] Negotiation successful
0811 12:19:19.385913 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":353,"threads_started":1}
I20260811 12:19:19.387365 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.367943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:56703)
0811 12:19:19.368918 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:19.368996 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:19.370802 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:19.371530 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:19.371540 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:19.372150 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:19:19.372790 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:19.372810 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.374027 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.374031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:19.375587 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:19.375603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.376628 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.376637 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:19.376873 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:19.377596 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:19:19.377625 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:19:19.380056 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:19:19.383111 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:19.383122 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:19.383142 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:19.383618 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:19.384012 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:19.384017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:19.384020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:19.384234 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:19.384831 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:19:19.384840 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:19:19.385273 (+ 433us) client_negotiation.cc:769] Sending connection context
0811 12:19:19.385562 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 12:19:19.386172 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":346,"threads_started":1}
I20260811 12:19:19.388921 486 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40833
I20260811 12:19:19.389303 486 heartbeater.cc:464] Registering TS with master...
I20260811 12:19:19.390111 486 heartbeater.cc:511] Master 127.7.39.62:40833 requested a full tablet report, sending...
I20260811 12:19:19.391763 32421 ts_manager.cc:194] Registered new tserver with Master: aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135)
I20260811 12:19:19.392858 32421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:56703
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:19:19.440917 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40833
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:19:19.786259 502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:19.786574 502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:19.786674 502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:19.794081 502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:19.794217 502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:19.794344 502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:19:19.804621 502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:19.806063 502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:19.807493 502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:19.821452 509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:19.821514 514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:19.821780 510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:19.822846 502 server_base.cc:1034] running on GCE node
I20260811 12:19:19.823657 502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:19.824723 502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:19.825937 502 hybrid_clock.cc:648] HybridClock initialized: now 1786450759825867 us; error 62 us; skew 500 ppm
I20260811 12:19:19.826263 502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:19 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450759, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:19.829535 502 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:19:19.831212 502 webserver.cc:460] Webserver started at http://127.7.39.3:43599/ using document root <none> and password file <none>
I20260811 12:19:19.831964 502 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:19.832093 502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:19.832422 502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:19:19.835629 502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "4ed3d2b7646e41729a509523090e6f55"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "61bda7038642eeb03a4f1967446143e9"
server_key_iv: "411659e00ca3bc57799d519ac322c008"
server_key_version: "encryptionkey@0"
I20260811 12:19:19.836410 502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "4ed3d2b7646e41729a509523090e6f55"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "61bda7038642eeb03a4f1967446143e9"
server_key_iv: "411659e00ca3bc57799d519ac322c008"
server_key_version: "encryptionkey@0"
I20260811 12:19:19.841930 502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:19:19.845716 524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:19.847136 502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 12:19:19.847301 502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "4ed3d2b7646e41729a509523090e6f55"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "61bda7038642eeb03a4f1967446143e9"
server_key_iv: "411659e00ca3bc57799d519ac322c008"
server_key_version: "encryptionkey@0"
I20260811 12:19:19.847498 502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:19.853315 32726 heartbeater.cc:503] Master 127.7.39.62:40833 was elected leader, sending a full tablet report...
I20260811 12:19:19.872658 502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:19.877255 502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:19.877516 502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:19.878496 502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:19:19.879845 502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:19:19.879920 502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:19.880012 502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:19:19.880055 502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:19.911880 502 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:33605
I20260811 12:19:19.911996 719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:33605 every 8 connection(s)
I20260811 12:19:19.913261 502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:19:19.922403 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 502
I20260811 12:19:19.922621 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:19:19.923213 7324 external_mini_cluster.cc:1428] Setting key 4b978d29ac68c49a1065334d6e4b69c3
Aug 11 12:19:19 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450759, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:19:19.934561 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.915959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:59199 (local address 127.7.39.62:40833)
0811 12:19:19.916496 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:19.916503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:19.917597 (+ 1094us) server_negotiation.cc:407] Connection header received
0811 12:19:19.919476 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:19.919481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:19.919557 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:19.919661 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:19.921252 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.922071 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.924289 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.924869 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.925006 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:19.928851 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:19.928857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:19.928860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:19.928898 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:19.931560 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:19.932376 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:19.932381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:19.932384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:19.932518 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:19.932918 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:19.932921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:19.932923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:19.933253 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:19.933364 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:19.934066 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 12:19:19.934283 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":245,"threads_started":1}
I20260811 12:19:19.935686 730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.916590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:59199)
0811 12:19:19.917404 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:19.917465 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:19.919199 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:19.919870 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:19.919879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:19.920418 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:19:19.921039 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:19.921058 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.922264 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.922270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:19.924148 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:19.924161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.925082 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.925089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:19.925409 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:19.926130 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:19:19.926162 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:19:19.928611 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:19:19.931701 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:19.931709 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:19.931726 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:19.932224 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:19.932619 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:19.932623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:19.932625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:19.932830 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:19.933366 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:19:19.933373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:19:19.933725 (+ 352us) client_negotiation.cc:769] Sending connection context
0811 12:19:19.933958 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 12:19:19.934397 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":291,"threads_started":1}
I20260811 12:19:19.937181 724 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40833
I20260811 12:19:19.937553 724 heartbeater.cc:464] Registering TS with master...
I20260811 12:19:19.938436 724 heartbeater.cc:511] Master 127.7.39.62:40833 requested a full tablet report, sending...
I20260811 12:19:19.939931 32421 ts_manager.cc:194] Registered new tserver with Master: 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605)
I20260811 12:19:19.940770 32421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:59199
I20260811 12:19:19.951736 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:19:19.969017 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.956509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34084 (local address 127.7.39.62:40833)
0811 12:19:19.956718 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:19.956723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:19.957132 (+ 409us) server_negotiation.cc:407] Connection header received
0811 12:19:19.957356 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:19.957360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:19.957464 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:19.957553 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:19.959013 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.959736 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.961273 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:19.961723 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:19.961819 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:19.964276 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:19.964281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:19.964284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:19.964318 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:19.966741 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:19.967410 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:19.967414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:19.967416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:19.967556 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:19.967849 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:19.967852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:19.967854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:19.968162 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:19.968354 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:19.968594 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 12:19:19.968773 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 12:19:19.973141 32421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:19:19.980124 32421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:19:20.015897 753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.992955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.7.39.3:33605)
0811 12:19:20.006358 (+ 13403us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.006367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.006404 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:19:20.006513 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.006522 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.006857 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.007078 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:20.008276 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.009999 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.013287 (+ 3288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.014475 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.014597 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.014865 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.015363 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.015536 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13142,"thread_start_us":12963,"threads_started":1}
I20260811 12:19:20.016096 746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.992205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:48468)
0811 12:19:20.004403 (+ 12198us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:20.004486 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:20.004763 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:20.007745 (+ 2982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:20.007748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:20.007774 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:20.008185 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.008199 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.010196 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.010201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.013043 (+ 2842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.013059 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.015209 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.015217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.015387 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.015419 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:19:20.015505 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:19:20.015657 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11851,"mutex_wait_us":100,"thread_start_us":14044,"threads_started":1}
I20260811 12:19:20.016278 747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.992461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.7.39.1:38405)
0811 12:19:20.007154 (+ 14693us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.007163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.007193 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:19:20.007291 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.007298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.007534 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.007744 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:20.008390 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.010026 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.013252 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.014684 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.014817 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.015056 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.015806 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.015975 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14504,"thread_start_us":14347,"threads_started":1}
I20260811 12:19:20.016713 749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.992293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33605 (local address 127.0.0.1:39750)
0811 12:19:20.005831 (+ 13538us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:20.005852 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:20.005993 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:20.007533 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:20.007540 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:20.007587 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:20.008086 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.008100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.010204 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.010208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.013042 (+ 2834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.013059 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.014975 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.014980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.015123 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.015172 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 12:19:20.015290 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 12:19:20.015502 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13412,"mutex_wait_us":179,"thread_start_us":11565,"threads_started":1}
I20260811 12:19:20.016888 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.992625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33135 (local address 127.0.0.1:51428)
0811 12:19:20.006500 (+ 13875us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:20.006518 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:20.008043 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:20.008651 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:20.008655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:20.008678 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:20.008989 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.008998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.010963 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.010966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.012681 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.012695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.016070 (+ 3375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.016079 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.016251 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.016282 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:19:20.016370 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:19:20.016492 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13751,"mutex_wait_us":90,"thread_start_us":12885,"threads_started":1}
I20260811 12:19:20.017486 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:19.993169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.7.39.2:33135)
0811 12:19:20.005312 (+ 12143us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.005323 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.006614 (+ 1291us) server_negotiation.cc:407] Connection header received
0811 12:19:20.008258 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.008267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.008469 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.008641 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:20.009143 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.010794 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.012843 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.015613 (+ 2770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.015726 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.015912 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.017079 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.017250 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11922,"thread_start_us":11772,"threads_started":1}
I20260811 12:19:20.020951 377 tablet_service.cc:1467] Processing CreateTablet for tablet 522bf4d815394970a2eb4fbee033723e (DEFAULT_TABLE table=test-table [id=6637a42463b74ae9b6cb39aa3e1bccf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:19:20.020962 32630 tablet_service.cc:1467] Processing CreateTablet for tablet 522bf4d815394970a2eb4fbee033723e (DEFAULT_TABLE table=test-table [id=6637a42463b74ae9b6cb39aa3e1bccf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:19:20.020967 603 tablet_service.cc:1467] Processing CreateTablet for tablet 522bf4d815394970a2eb4fbee033723e (DEFAULT_TABLE table=test-table [id=6637a42463b74ae9b6cb39aa3e1bccf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:19:20.022362 377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 522bf4d815394970a2eb4fbee033723e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:19:20.022502 32630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 522bf4d815394970a2eb4fbee033723e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:19:20.022495 603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 522bf4d815394970a2eb4fbee033723e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:19:20.034622 758 tablet_bootstrap.cc:492] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Bootstrap starting.
I20260811 12:19:20.038646 758 tablet_bootstrap.cc:654] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Neither blocks nor log segments found. Creating new log.
I20260811 12:19:20.039767 758 log.cc:824] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:19:20.042519 759 tablet_bootstrap.cc:492] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Bootstrap starting.
I20260811 12:19:20.043126 758 tablet_bootstrap.cc:492] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: No bootstrap required, opened a new log
I20260811 12:19:20.043397 758 ts_tablet_manager.cc:1397] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260811 12:19:20.045358 760 tablet_bootstrap.cc:492] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Bootstrap starting.
I20260811 12:19:20.045734 759 tablet_bootstrap.cc:654] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Neither blocks nor log segments found. Creating new log.
I20260811 12:19:20.047173 759 log.cc:824] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Log is configured to *not* fsync() on all Append() calls
I20260811 12:19:20.048789 760 tablet_bootstrap.cc:654] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Neither blocks nor log segments found. Creating new log.
I20260811 12:19:20.050132 760 log.cc:824] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Log is configured to *not* fsync() on all Append() calls
I20260811 12:19:20.050678 759 tablet_bootstrap.cc:492] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: No bootstrap required, opened a new log
I20260811 12:19:20.051023 759 ts_tablet_manager.cc:1397] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:19:20.051745 758 raft_consensus.cc:348] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.052163 758 raft_consensus.cc:374] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:19:20.052269 758 raft_consensus.cc:729] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aab77284fd3b4b65926810ae8e50e95e, State: Initialized, Role: FOLLOWER
I20260811 12:19:20.052914 758 consensus_queue.cc:260] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.053589 760 tablet_bootstrap.cc:492] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: No bootstrap required, opened a new log
I20260811 12:19:20.053958 760 ts_tablet_manager.cc:1397] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260811 12:19:20.054253 486 heartbeater.cc:503] Master 127.7.39.62:40833 was elected leader, sending a full tablet report...
I20260811 12:19:20.054939 758 ts_tablet_manager.cc:1428] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Time spent starting tablet: real 0.011s user 0.008s sys 0.005s
I20260811 12:19:20.061844 759 raft_consensus.cc:348] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.062337 759 raft_consensus.cc:374] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:19:20.062445 759 raft_consensus.cc:729] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed3d2b7646e41729a509523090e6f55, State: Initialized, Role: FOLLOWER
I20260811 12:19:20.063093 759 consensus_queue.cc:260] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.064195 724 heartbeater.cc:503] Master 127.7.39.62:40833 was elected leader, sending a full tablet report...
I20260811 12:19:20.064246 760 raft_consensus.cc:348] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.064724 760 raft_consensus.cc:374] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:19:20.064833 760 raft_consensus.cc:729] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b0929c4ce2433eaef580e795381680, State: Initialized, Role: FOLLOWER
I20260811 12:19:20.065011 759 ts_tablet_manager.cc:1428] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 12:19:20.065472 760 consensus_queue.cc:260] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.067147 760 ts_tablet_manager.cc:1428] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260811 12:19:20.073524 32733 tablet.cc:2369] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:19:20.116636 488 tablet.cc:2369] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:19:20.131528 766 raft_consensus.cc:482] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:19:20.131875 766 raft_consensus.cc:504] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.133715 766 leader_election.cc:288] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405), aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135)
I20260811 12:19:20.139367 730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:20.134113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33135 (local address 127.7.39.3:54521)
0811 12:19:20.134328 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:20.134348 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:20.134456 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:20.134796 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:20.134801 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:20.134830 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:20.135018 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.135027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.136119 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.136123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.137482 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.137498 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.138611 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.138616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.138793 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.138820 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 12:19:20.138929 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 12:19:20.139069 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":17}
I20260811 12:19:20.139367 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:20.134245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:54521 (local address 127.7.39.2:33135)
0811 12:19:20.134440 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.134445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.134462 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:19:20.134599 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.134603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.134668 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.134751 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:20.135165 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.135967 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.137648 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.138435 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.138566 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.138679 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.138973 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.139107 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:19:20.140666 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:20.134117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.7.39.3:33997)
0811 12:19:20.135340 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:20.135354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:20.135474 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:20.135794 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:20.135798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:20.135823 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:20.136183 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.136194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.137244 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.137248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.138696 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:20.138710 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.140076 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.140080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:20.140209 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.140233 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:19:20.140305 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:19:20.140396 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"mutex_wait_us":466,"thread_start_us":390,"threads_started":1}
I20260811 12:19:20.140729 747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:20.134123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:33997 (local address 127.7.39.1:38405)
0811 12:19:20.134336 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.134343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.135448 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 12:19:20.135611 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.135617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.135685 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.135761 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:20.136342 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.137072 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.138890 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.139876 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.140027 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.140136 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.140352 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.140487 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 12:19:20.140552 411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "522bf4d815394970a2eb4fbee033723e" candidate_uuid: "4ed3d2b7646e41729a509523090e6f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aab77284fd3b4b65926810ae8e50e95e" is_pre_election: true
I20260811 12:19:20.141237 411 raft_consensus.cc:2454] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed3d2b7646e41729a509523090e6f55 in term 0.
I20260811 12:19:20.141979 532 leader_election.cc:302] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed3d2b7646e41729a509523090e6f55, aab77284fd3b4b65926810ae8e50e95e; no voters:
I20260811 12:19:20.141865 32660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "522bf4d815394970a2eb4fbee033723e" candidate_uuid: "4ed3d2b7646e41729a509523090e6f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b0929c4ce2433eaef580e795381680" is_pre_election: true
I20260811 12:19:20.142452 766 raft_consensus.cc:2781] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:19:20.142449 32660 raft_consensus.cc:2454] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed3d2b7646e41729a509523090e6f55 in term 0.
I20260811 12:19:20.142619 766 raft_consensus.cc:482] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:19:20.142724 766 raft_consensus.cc:3037] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:19:20.145051 766 raft_consensus.cc:504] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.145757 766 leader_election.cc:288] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [CANDIDATE]: Term 1 election: Requested vote from peers 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405), aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135)
I20260811 12:19:20.146253 32660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "522bf4d815394970a2eb4fbee033723e" candidate_uuid: "4ed3d2b7646e41729a509523090e6f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b0929c4ce2433eaef580e795381680"
I20260811 12:19:20.146351 411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "522bf4d815394970a2eb4fbee033723e" candidate_uuid: "4ed3d2b7646e41729a509523090e6f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aab77284fd3b4b65926810ae8e50e95e"
I20260811 12:19:20.146456 32660 raft_consensus.cc:3037] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:19:20.146580 411 raft_consensus.cc:3037] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:19:20.148797 32660 raft_consensus.cc:2454] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed3d2b7646e41729a509523090e6f55 in term 1.
I20260811 12:19:20.148855 411 raft_consensus.cc:2454] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed3d2b7646e41729a509523090e6f55 in term 1.
I20260811 12:19:20.149309 526 leader_election.cc:302] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed3d2b7646e41729a509523090e6f55, 57b0929c4ce2433eaef580e795381680; no voters:
I20260811 12:19:20.149657 766 raft_consensus.cc:2781] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:19:20.150161 766 raft_consensus.cc:686] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 1 LEADER]: Becoming Leader. State: Replica: 4ed3d2b7646e41729a509523090e6f55, State: Running, Role: LEADER
I20260811 12:19:20.150705 766 consensus_queue.cc:237] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } }
I20260811 12:19:20.155639 32421 catalog_manager.cc:5495] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3). New cstate: current_term: 1 leader_uuid: "4ed3d2b7646e41729a509523090e6f55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } health_report { overall_health: UNKNOWN } } }
W20260811 12:19:20.164904 726 tablet.cc:2369] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:19:20.173369 753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:20.167960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.7.39.3:33605)
0811 12:19:20.168167 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.168171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.168190 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:19:20.168375 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.168379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.168466 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.168598 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:19:20.168931 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.169736 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.171299 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.171702 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.171793 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.172276 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:19:20.172819 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:19:20.172928 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.173003 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.173076 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:19:20.181895 32660 raft_consensus.cc:1261] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Refusing update from remote peer 4ed3d2b7646e41729a509523090e6f55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:19:20.181887 411 raft_consensus.cc:1261] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Refusing update from remote peer 4ed3d2b7646e41729a509523090e6f55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:19:20.182868 766 consensus_queue.cc:1035] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:19:20.183184 773 consensus_queue.cc:1035] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:19:20.194108 776 mvcc.cc:205] Tried to move back new op lower bound from 7317302313693831168 to 7317302313581699072. Current Snapshot: MvccSnapshot[applied={T|T < 7317302313693831168}]
I20260811 12:19:20.194726 778 mvcc.cc:205] Tried to move back new op lower bound from 7317302313693831168 to 7317302313581699072. Current Snapshot: MvccSnapshot[applied={T|T < 7317302313693831168}]
I20260811 12:19:20.202191 779 mvcc.cc:205] Tried to move back new op lower bound from 7317302313693831168 to 7317302313581699072. Current Snapshot: MvccSnapshot[applied={T|T < 7317302313693831168}]
I20260811 12:19:20.210224 32421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:19:20.210544 32421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:19:20.216398 32421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Sending DeleteTablet for 3 replicas of tablet 522bf4d815394970a2eb4fbee033723e
I20260811 12:19:20.218031 32630 tablet_service.cc:1514] Processing DeleteTablet for tablet 522bf4d815394970a2eb4fbee033723e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:19:20 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:48468
I20260811 12:19:20.218784 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 32522
I20260811 12:19:20.218835 377 tablet_service.cc:1514] Processing DeleteTablet for tablet 522bf4d815394970a2eb4fbee033723e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:19:20 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:51428
I20260811 12:19:20.219026 603 tablet_service.cc:1514] Processing DeleteTablet for tablet 522bf4d815394970a2eb4fbee033723e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:19:20 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:39750
I20260811 12:19:20.242887 796 tablet_replica.cc:330] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: stopping tablet replica
I20260811 12:19:20.243695 795 tablet_replica.cc:330] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: stopping tablet replica
I20260811 12:19:20.244305 796 raft_consensus.cc:2229] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:19:20.244863 795 raft_consensus.cc:2229] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:19:20.245527 796 raft_consensus.cc:2258] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:19:20.247037 795 raft_consensus.cc:2258] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:19:20.249826 795 ts_tablet_manager.cc:1905] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:19:20.251333 796 ts_tablet_manager.cc:1905] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:19:20.255448 797 tablet_replica.cc:330] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: stopping tablet replica
I20260811 12:19:20.256042 797 raft_consensus.cc:2229] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:19:20.258486 797 raft_consensus.cc:2258] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:19:20.260638 796 ts_tablet_manager.cc:1918] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:19:20.260627 795 ts_tablet_manager.cc:1918] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:19:20.260847 796 log.cc:1196] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/522bf4d815394970a2eb4fbee033723e
I20260811 12:19:20.260970 795 log.cc:1196] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/522bf4d815394970a2eb4fbee033723e
I20260811 12:19:20.261466 796 ts_tablet_manager.cc:1939] T 522bf4d815394970a2eb4fbee033723e P aab77284fd3b4b65926810ae8e50e95e: Deleting consensus metadata
I20260811 12:19:20.261679 795 ts_tablet_manager.cc:1939] T 522bf4d815394970a2eb4fbee033723e P 57b0929c4ce2433eaef580e795381680: Deleting consensus metadata
I20260811 12:19:20.263609 32401 catalog_manager.cc:4841] TS aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135): tablet 522bf4d815394970a2eb4fbee033723e (table test-table [id=6637a42463b74ae9b6cb39aa3e1bccf5]) successfully deleted
I20260811 12:19:20.263686 797 ts_tablet_manager.cc:1905] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:19:20.263556 32398 catalog_manager.cc:4841] TS 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405): tablet 522bf4d815394970a2eb4fbee033723e (table test-table [id=6637a42463b74ae9b6cb39aa3e1bccf5]) successfully deleted
Aug 11 12:19:20 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:19:20.270512 797 ts_tablet_manager.cc:1918] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:19:20.270784 797 log.cc:1196] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/522bf4d815394970a2eb4fbee033723e
I20260811 12:19:20.271319 797 ts_tablet_manager.cc:1939] T 522bf4d815394970a2eb4fbee033723e P 4ed3d2b7646e41729a509523090e6f55: Deleting consensus metadata
I20260811 12:19:20.273043 32400 catalog_manager.cc:4841] TS 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605): tablet 522bf4d815394970a2eb4fbee033723e (table test-table [id=6637a42463b74ae9b6cb39aa3e1bccf5]) successfully deleted
I20260811 12:19:20.273850 747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:20.220235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.7.39.1:38405)
0811 12:19:20.220433 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.220440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.246369 (+ 25929us) server_negotiation.cc:407] Connection header received
0811 12:19:20.246933 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.246940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.247071 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.247156 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:20.252057 (+ 4901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.253378 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.255765 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.256461 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.256580 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.265521 (+ 8941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:20.265540 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:20.265561 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:20.265620 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:20.269572 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:20.270684 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:20.270694 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:20.270697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:20.270890 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:20.271453 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:20.271460 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:20.271463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:20.271983 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:20.272306 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.273077 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.273417 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:19:20.275449 32720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:20.365936 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 32522
I20260811 12:19:20.396826 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 32744
Aug 11 12:19:20 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:19:20.413838 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:20.397671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.7.39.2:33135)
0811 12:19:20.397865 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.397872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.397950 (+ 78us) server_negotiation.cc:407] Connection header received
0811 12:19:20.398146 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.398151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.398237 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.398324 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:20.399672 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.400433 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.401866 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.402274 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.402386 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.406994 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:20.407015 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:20.407039 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:20.407096 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:20.410862 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:20.411582 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:20.411590 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:20.411594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:20.411743 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:20.412099 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:20.412103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:20.412106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:20.412611 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:20.412921 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.413225 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.413460 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:19:20.414534 477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:20.504550 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 32744
I20260811 12:19:20.534078 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 502
Aug 11 12:19:20 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:19:20.555842 753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:20.534866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39772 (local address 127.7.39.3:33605)
0811 12:19:20.535037 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:20.535043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:20.535128 (+ 85us) server_negotiation.cc:407] Connection header received
0811 12:19:20.535342 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:20.535346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:20.535408 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:20.535490 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:20.537360 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.538167 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.540195 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:20.540663 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:20.540775 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:20.546661 (+ 5886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:20.546688 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:20.546712 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:20.546776 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:20.551154 (+ 4378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:20.552422 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:20.552433 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:20.552437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:20.552653 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:20.553123 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:20.553128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:20.553132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:20.553646 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:20.553919 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:20.555155 (+ 1236us) server_negotiation.cc:299] Negotiation successful
0811 12:19:20.555397 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:19:20.558068 714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:20.685307 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 502
I20260811 12:19:20.707844 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 32369
I20260811 12:19:20.709206 32468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:20.820998 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 32369
I20260811 12:19:40.847985 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40833
--webserver_interface=127.7.39.62
--webserver_port=45785
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:40833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:19:41.204509 809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:41.204850 809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:41.204945 809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:41.212494 809 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:19:41.212625 809 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:19:41.212663 809 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:19:41.212719 809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:19:41.212775 809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:41.212844 809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:41.212891 809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:19:41.212946 809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:19:41.223430 809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:40833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=45785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:41.224824 809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:41.226250 809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:41.253459 819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:41.253536 820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:41.253623 822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:41.253525 809 server_base.cc:1034] running on GCE node
I20260811 12:19:41.255111 809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:41.256844 809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:41.258163 809 hybrid_clock.cc:648] HybridClock initialized: now 1786450781258082 us; error 49 us; skew 500 ppm
I20260811 12:19:41.258622 809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:41 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450781, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:41.267876 809 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:19:41.269872 809 webserver.cc:460] Webserver started at http://127.7.39.62:45785/ using document root <none> and password file <none>
I20260811 12:19:41.270733 809 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:41.270879 809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:41.278227 809 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:19:41.281848 840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:41.283600 809 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:19:41.283845 809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "3cb90125183c4b5a99b12499648bebda"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "3979d9da92e985efbf327800de55442f"
server_key_iv: "a740aa4b2b873f67c4df21426138cbf7"
server_key_version: "encryptionkey@0"
I20260811 12:19:41.285130 809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:41.299809 809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:41.304657 809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:41.304977 809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:41.325490 809 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:40833
I20260811 12:19:41.325512 918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:40833 every 8 connection(s)
I20260811 12:19:41.327211 809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:19:41.331017 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 809
I20260811 12:19:41.331878 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:38405
--local_ip_for_outbound_sockets=127.7.39.1
--tserver_master_addrs=127.7.39.62:40833
--webserver_port=41983
--webserver_interface=127.7.39.1
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:19:41.335022 919 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:19:41.338660 919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Bootstrap starting.
I20260811 12:19:41.359450 919 log.cc:824] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Log is configured to *not* fsync() on all Append() calls
I20260811 12:19:41.372566 919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:19:41.373198 919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Bootstrap complete.
I20260811 12:19:41.382584 919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:19:41.383538 919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb90125183c4b5a99b12499648bebda, State: Initialized, Role: FOLLOWER
I20260811 12:19:41.384320 919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:19:41.384586 919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:19:41.384712 919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:19:41.384879 919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:19:41.388900 919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:19:41.389540 919 leader_election.cc:302] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cb90125183c4b5a99b12499648bebda; no voters:
I20260811 12:19:41.390228 919 leader_election.cc:288] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:19:41.390506 927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:19:41.391450 927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 2 LEADER]: Becoming Leader. State: Replica: 3cb90125183c4b5a99b12499648bebda, State: Running, Role: LEADER
I20260811 12:19:41.392134 927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:19:41.392480 919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:19:41.394764 928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3cb90125183c4b5a99b12499648bebda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } } }
I20260811 12:19:41.395180 928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: This master's current role is: LEADER
I20260811 12:19:41.402416 929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cb90125183c4b5a99b12499648bebda. Latest consensus state: current_term: 2 leader_uuid: "3cb90125183c4b5a99b12499648bebda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } } }
I20260811 12:19:41.403337 929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: This master's current role is: LEADER
I20260811 12:19:41.403646 935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:19:41.412961 935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:19:41.414728 935 catalog_manager.cc:1259] Loaded cluster ID: add29195478a495c8454f2ea4effcf90
I20260811 12:19:41.414932 935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:19:41.419468 935 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:19:41.421475 935 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Loaded TSK: 0
I20260811 12:19:41.456034 952 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:19:41.459012 935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Generated new TSK 1
I20260811 12:19:41.459488 935 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:19:41.709313 921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:41.709797 921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:41.709978 921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:41.717908 921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:41.718052 921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:41.718205 921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:19:41.729291 921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:38405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=41983
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:41.730736 921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:41.732188 921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:41.747516 964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:41.747769 969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:41.747987 963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:41.748471 921 server_base.cc:1034] running on GCE node
I20260811 12:19:41.749111 921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:41.750167 921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:41.751382 921 hybrid_clock.cc:648] HybridClock initialized: now 1786450781751319 us; error 58 us; skew 500 ppm
I20260811 12:19:41.751734 921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:41 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450781, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:41.755379 921 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:19:41.757190 921 webserver.cc:460] Webserver started at http://127.7.39.1:41983/ using document root <none> and password file <none>
I20260811 12:19:41.758009 921 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:41.758169 921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:41.764633 921 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 12:19:41.767961 983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:41.769739 921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:19:41.769953 921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "57b0929c4ce2433eaef580e795381680"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "f9a2ec521d7eb2c476b5d2209d604dbe"
server_key_iv: "42a7eb60dfbfbd3fbb164e6c481c8eb4"
server_key_version: "encryptionkey@0"
I20260811 12:19:41.771001 921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:41.788172 921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:41.793004 921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:41.793313 921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:41.794529 921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:19:41.796056 921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:19:41.796160 921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:41.796268 921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:19:41.796339 921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:41.826161 921 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:38405
I20260811 12:19:41.826215 1168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:38405 every 8 connection(s)
I20260811 12:19:41.827675 921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:19:41.833885 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 921
I20260811 12:19:41.834568 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:33135
--local_ip_for_outbound_sockets=127.7.39.2
--tserver_master_addrs=127.7.39.62:40833
--webserver_port=45583
--webserver_interface=127.7.39.2
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:19:41 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450781, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:19:41.853432 1172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:41.830957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:51491 (local address 127.7.39.62:40833)
0811 12:19:41.831611 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:41.831626 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:41.832413 (+ 787us) server_negotiation.cc:407] Connection header received
0811 12:19:41.834870 (+ 2457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:41.834891 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:41.835301 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:41.835718 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:41.837611 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:41.838972 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:41.840670 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:41.841427 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:41.841648 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:41.845404 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:41.845413 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:41.845431 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:41.845489 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:41.848481 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:41.849345 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:41.849354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:41.849360 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:41.849549 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:41.850007 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:41.850010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:41.850013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:41.850554 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:41.850754 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:41.851611 (+ 857us) server_negotiation.cc:299] Negotiation successful
0811 12:19:41.852027 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":297,"threads_started":1}
I20260811 12:19:41.853462 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:41.831246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:51491)
0811 12:19:41.832218 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:41.832270 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:41.834197 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:41.835921 (+ 1724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:41.835934 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:41.836649 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:19:41.837355 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:41.837381 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:41.839176 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:41.839181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:41.840493 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:41.840509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:41.841689 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:41.841697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:41.841985 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:41.842741 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:19:41.842766 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:19:41.845066 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:19:41.848650 (+ 3584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:41.848661 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:41.848678 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:41.849192 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:41.849669 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:41.849674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:41.849678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:41.849876 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:41.850700 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:19:41.850709 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:19:41.851079 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 12:19:41.851333 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 12:19:41.851949 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":382,"threads_started":1}
I20260811 12:19:41.856624 1169 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40833
I20260811 12:19:41.856945 1169 heartbeater.cc:464] Registering TS with master...
I20260811 12:19:41.857782 1169 heartbeater.cc:511] Master 127.7.39.62:40833 requested a full tablet report, sending...
I20260811 12:19:41.860028 869 ts_manager.cc:194] Registered new tserver with Master: 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405)
I20260811 12:19:41.861740 869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:51491
W20260811 12:19:42.180994 1175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:42.181391 1175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:42.181552 1175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:42.188995 1175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:42.189138 1175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:42.189327 1175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:19:42.199682 1175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:33135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=45583
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:42.201184 1175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:42.202637 1175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:42.216477 1182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:42.216449 1183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:42.216641 1189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:42.217363 1175 server_base.cc:1034] running on GCE node
I20260811 12:19:42.218012 1175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:42.218961 1175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:42.220170 1175 hybrid_clock.cc:648] HybridClock initialized: now 1786450782220121 us; error 45 us; skew 500 ppm
I20260811 12:19:42.220476 1175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:42 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450782, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:42.223734 1175 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:19:42.225231 1175 webserver.cc:460] Webserver started at http://127.7.39.2:45583/ using document root <none> and password file <none>
I20260811 12:19:42.225934 1175 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:42.226053 1175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:42.231552 1175 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:19:42.234449 1199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:42.236011 1175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:19:42.236198 1175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "aab77284fd3b4b65926810ae8e50e95e"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "4ff0b7575311fa65b83094b61acde690"
server_key_iv: "2310cc19f749d003a7dd70e7fda6ffac"
server_key_version: "encryptionkey@0"
I20260811 12:19:42.237337 1175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:42.257789 1175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:42.262071 1175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:42.262333 1175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:42.263365 1175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:19:42.264838 1175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:19:42.264930 1175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:42.265044 1175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:19:42.265113 1175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:42.297427 1175 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:33135
I20260811 12:19:42.297518 1405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:33135 every 8 connection(s)
I20260811 12:19:42.299021 1175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:19:42.305729 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 1175
I20260811 12:19:42.306394 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:33605
--local_ip_for_outbound_sockets=127.7.39.3
--tserver_master_addrs=127.7.39.62:40833
--webserver_port=43599
--webserver_interface=127.7.39.3
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:19:42 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450782, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:19:42.321074 1172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.301787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:43141 (local address 127.7.39.62:40833)
0811 12:19:42.302039 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:42.302044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:42.303318 (+ 1274us) server_negotiation.cc:407] Connection header received
0811 12:19:42.305380 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:42.305398 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:42.305508 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:42.305603 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:42.307404 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.308134 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.309958 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.310397 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.310524 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.314127 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:42.314133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:42.314138 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:42.314182 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:42.317391 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:42.318430 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:42.318435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:42.318438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:42.318598 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:42.319111 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:42.319115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:42.319117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:42.319425 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:42.319586 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:42.320551 (+ 965us) server_negotiation.cc:299] Negotiation successful
0811 12:19:42.320822 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:19:42.322605 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.302291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:43141)
0811 12:19:42.303118 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:42.303163 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:42.305039 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:42.305897 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:42.305911 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:42.306508 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:19:42.307207 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.307227 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.308313 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.308317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.309804 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.309817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.310657 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.310665 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.310895 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.311577 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:19:42.311606 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:19:42.313841 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:19:42.317627 (+ 3786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:42.317637 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:42.317655 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:42.318237 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:42.318730 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:42.318734 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:42.318736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:42.318968 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:42.319571 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:19:42.319581 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:19:42.320103 (+ 522us) client_negotiation.cc:769] Sending connection context
0811 12:19:42.320435 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 12:19:42.321037 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":331,"threads_started":1}
I20260811 12:19:42.324290 1406 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40833
I20260811 12:19:42.324635 1406 heartbeater.cc:464] Registering TS with master...
I20260811 12:19:42.325500 1406 heartbeater.cc:511] Master 127.7.39.62:40833 requested a full tablet report, sending...
I20260811 12:19:42.327044 869 ts_manager.cc:194] Registered new tserver with Master: aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135)
I20260811 12:19:42.327891 869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:43141
W20260811 12:19:42.668841 1409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:19:42.669142 1409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:19:42.669284 1409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:19:42.676723 1409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:19:42.676856 1409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:19:42.676975 1409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:19:42.687507 1409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:33605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=43599
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:19:42.688985 1409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:19:42.690428 1409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:19:42.703800 1419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:42.703773 1425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:19:42.703830 1422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:19:42.704380 1409 server_base.cc:1034] running on GCE node
I20260811 12:19:42.705109 1409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:19:42.706146 1409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:19:42.707352 1409 hybrid_clock.cc:648] HybridClock initialized: now 1786450782707287 us; error 64 us; skew 500 ppm
I20260811 12:19:42.707692 1409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:19:42 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450782, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:19:42.710906 1409 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:19:42.712446 1409 webserver.cc:460] Webserver started at http://127.7.39.3:43599/ using document root <none> and password file <none>
I20260811 12:19:42.713131 1409 fs_manager.cc:359] Metadata directory not provided
I20260811 12:19:42.713302 1409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:19:42.719393 1409 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:19:42.722422 1437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:42.723948 1409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:19:42.724129 1409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "4ed3d2b7646e41729a509523090e6f55"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "61bda7038642eeb03a4f1967446143e9"
server_key_iv: "411659e00ca3bc57799d519ac322c008"
server_key_version: "encryptionkey@0"
I20260811 12:19:42.725242 1409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:19:42.741434 1409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:19:42.746086 1409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:19:42.746342 1409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:19:42.747344 1409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:19:42.748726 1409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:19:42.748831 1409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:42.748914 1409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:19:42.748981 1409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:19:42.782677 1409 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:33605
I20260811 12:19:42.782793 1629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:33605 every 8 connection(s)
I20260811 12:19:42.784272 1409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:19:42.793365 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 1409
Aug 11 12:19:42 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450782, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:19:42.806959 1172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.787141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:60897 (local address 127.7.39.62:40833)
0811 12:19:42.787327 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:42.787331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:42.788731 (+ 1400us) server_negotiation.cc:407] Connection header received
0811 12:19:42.790708 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:42.790713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:42.790791 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:42.790883 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:42.792512 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.793339 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.795528 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.796056 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.796161 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.800101 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:42.800106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:42.800110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:42.800167 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:42.803366 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:42.804280 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:42.804285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:42.804287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:42.804434 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:42.804974 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:42.804979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:42.804983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:42.805383 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:42.805511 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:42.806471 (+ 960us) server_negotiation.cc:299] Negotiation successful
0811 12:19:42.806684 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:19:42.808485 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.787738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:60897)
0811 12:19:42.788511 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:42.788563 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:42.790405 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:42.791097 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:42.791106 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:42.791684 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:19:42.792371 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.792394 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.793518 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.793524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.795390 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.795404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.796332 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.796340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.796684 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.797487 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:19:42.797520 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:19:42.799857 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:19:42.803534 (+ 3677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:42.803542 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:42.803558 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:42.804168 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:42.804548 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:19:42.804554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:19:42.804558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:19:42.804805 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:19:42.805538 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:19:42.805548 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:19:42.806028 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 12:19:42.806378 (+ 350us) client_negotiation.cc:240] Negotiation successful
0811 12:19:42.806968 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":295,"threads_started":1}
I20260811 12:19:42.810251 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.794471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.7.39.62:40833)
0811 12:19:42.795319 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:42.795325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:42.795343 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:19:42.795388 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:42.795392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:42.795448 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:42.795552 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:42.797608 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.798367 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.799892 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.800436 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.800534 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.803938 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:42.803946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:42.803951 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:42.803991 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:42.807157 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:42.808018 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:42.808022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:42.808024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:42.808145 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:42.808649 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:42.808655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:42.808659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:42.808994 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:42.809319 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:42.809637 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 12:19:42.809861 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":568,"threads_started":1}
I20260811 12:19:42.810338 1632 heartbeater.cc:347] Connected to a master server at 127.7.39.62:40833
I20260811 12:19:42.810704 1632 heartbeater.cc:464] Registering TS with master...
I20260811 12:19:42.811518 1632 heartbeater.cc:511] Master 127.7.39.62:40833 requested a full tablet report, sending...
I20260811 12:19:42.813090 869 ts_manager.cc:194] Registered new tserver with Master: 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605)
I20260811 12:19:42.813923 869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:60897
I20260811 12:19:42.825309 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:19:42.832806 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.826417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.7.39.62:40833)
0811 12:19:42.826618 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:42.826624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:42.826991 (+ 367us) server_negotiation.cc:407] Connection header received
0811 12:19:42.827238 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:42.827242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:42.827393 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:42.827515 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:19:42.827998 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.828782 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.830902 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.831332 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.831424 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.831969 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:19:42.832282 (+ 313us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 12:19:42.832578 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38280: token expired
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 12:19:42.832952 1636 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38280: token expired
W20260811 12:19:42.833609 734 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.39.62:40833)
I20260811 12:19:42.847038 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.834143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.7.39.62:40833)
0811 12:19:42.834312 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:42.834317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:42.834337 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:19:42.834495 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:42.834501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:42.834563 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:42.834649 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:42.835999 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.836774 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.838264 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.838701 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.838815 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.841684 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:42.841690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:42.841693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:42.841732 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:42.844156 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:42.844891 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:42.844896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:42.844898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:42.845036 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:42.845510 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:42.845516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:42.845519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:42.845867 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:42.846100 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:42.846465 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 12:19:42.846710 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:19:42.850700 869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:19:42.858217 869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:19:42.864984 1169 heartbeater.cc:503] Master 127.7.39.62:40833 was elected leader, sending a full tablet report...
I20260811 12:19:42.895093 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.871657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46258 (local address 127.7.39.3:33605)
0811 12:19:42.885475 (+ 13818us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:42.885495 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:42.886932 (+ 1437us) server_negotiation.cc:407] Connection header received
0811 12:19:42.887103 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:42.887111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:42.887346 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:42.887547 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:42.888334 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.889931 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.892337 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.893808 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.893941 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.894191 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:42.894522 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 12:19:42.894755 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12157,"thread_start_us":11971,"threads_started":1}
I20260811 12:19:42.895224 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.870970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33605 (local address 127.0.0.1:46258)
0811 12:19:42.886789 (+ 15819us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:42.886807 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:42.886950 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:42.887660 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:42.887665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:42.887688 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:42.888124 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.888136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.890141 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.890145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.892176 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.892191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.894025 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.894032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.894247 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.894301 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 12:19:42.894453 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 12:19:42.894669 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15677,"mutex_wait_us":152,"thread_start_us":15536,"threads_started":1}
I20260811 12:19:42.895355 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.870573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33135 (local address 127.0.0.1:54816)
0811 12:19:42.885991 (+ 15418us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:42.886012 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:42.886152 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:42.887102 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:42.887107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:42.887131 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:42.888001 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.888026 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.890132 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.890138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.892798 (+ 2660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.892808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.894554 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.894560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.894750 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.894782 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:19:42.894877 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:19:42.895007 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15292,"mutex_wait_us":248,"thread_start_us":15124,"threads_started":1}
I20260811 12:19:42.896044 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.871261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.7.39.2:33135)
0811 12:19:42.882321 (+ 11060us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:42.882333 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:42.886313 (+ 3980us) server_negotiation.cc:407] Connection header received
0811 12:19:42.886452 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:42.886462 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:42.886789 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:42.887034 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:42.888257 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.889963 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.892960 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.894078 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.894204 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.894396 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:42.895555 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0811 12:19:42.895729 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10896,"thread_start_us":10773,"threads_started":1}
I20260811 12:19:42.896550 1645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.870355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50706)
0811 12:19:42.883126 (+ 12771us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:42.883178 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:42.883378 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:42.887078 (+ 3700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:42.887086 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:42.887131 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:42.888056 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.888070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.889971 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.889977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.892740 (+ 2763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:42.892760 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.894922 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.894929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:42.895221 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.895290 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 12:19:42.895400 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 12:19:42.896128 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12438,"mutex_wait_us":219,"spinlock_wait_cycles":233728,"thread_start_us":12281,"threads_started":1}
I20260811 12:19:42.897261 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:42.870643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.7.39.1:38405)
0811 12:19:42.886433 (+ 15790us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:42.886441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:42.886467 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:19:42.886551 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:42.886558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:42.886786 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:42.887381 (+ 595us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:42.888280 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.889814 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.892933 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:42.894399 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:42.894526 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:42.894747 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:42.896824 (+ 2077us) server_negotiation.cc:299] Negotiation successful
0811 12:19:42.896973 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15601,"thread_start_us":15431,"threads_started":1}
I20260811 12:19:42.900208 1516 tablet_service.cc:1467] Processing CreateTablet for tablet 17f962e23b884d97b20600a9ef5a80ad (DEFAULT_TABLE table=test-table [id=2101f54d75a243d9a92e92d4ae26f543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:19:42.900208 1297 tablet_service.cc:1467] Processing CreateTablet for tablet 17f962e23b884d97b20600a9ef5a80ad (DEFAULT_TABLE table=test-table [id=2101f54d75a243d9a92e92d4ae26f543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:19:42.901829 1297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f962e23b884d97b20600a9ef5a80ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:19:42.901829 1516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f962e23b884d97b20600a9ef5a80ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:19:42.904975 1068 tablet_service.cc:1467] Processing CreateTablet for tablet 17f962e23b884d97b20600a9ef5a80ad (DEFAULT_TABLE table=test-table [id=2101f54d75a243d9a92e92d4ae26f543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:19:42.906375 1068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f962e23b884d97b20600a9ef5a80ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:19:42.918442 1658 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Bootstrap starting.
I20260811 12:19:42.919414 1659 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Bootstrap starting.
I20260811 12:19:42.920667 1661 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Bootstrap starting.
I20260811 12:19:42.923811 1661 tablet_bootstrap.cc:654] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Neither blocks nor log segments found. Creating new log.
I20260811 12:19:42.924285 1659 tablet_bootstrap.cc:654] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Neither blocks nor log segments found. Creating new log.
I20260811 12:19:42.924571 1658 tablet_bootstrap.cc:654] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Neither blocks nor log segments found. Creating new log.
I20260811 12:19:42.925035 1661 log.cc:824] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Log is configured to *not* fsync() on all Append() calls
I20260811 12:19:42.925657 1659 log.cc:824] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:19:42.925741 1658 log.cc:824] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Log is configured to *not* fsync() on all Append() calls
I20260811 12:19:42.927917 1661 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: No bootstrap required, opened a new log
I20260811 12:19:42.928182 1661 ts_tablet_manager.cc:1397] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:19:42.928725 1659 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: No bootstrap required, opened a new log
I20260811 12:19:42.928999 1659 ts_tablet_manager.cc:1397] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 12:19:42.930383 1658 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: No bootstrap required, opened a new log
I20260811 12:19:42.930652 1658 ts_tablet_manager.cc:1397] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260811 12:19:42.936028 1659 raft_consensus.cc:348] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:42.936488 1659 raft_consensus.cc:374] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:19:42.936592 1659 raft_consensus.cc:729] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aab77284fd3b4b65926810ae8e50e95e, State: Initialized, Role: FOLLOWER
I20260811 12:19:42.937362 1659 consensus_queue.cc:260] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:42.938661 1406 heartbeater.cc:503] Master 127.7.39.62:40833 was elected leader, sending a full tablet report...
I20260811 12:19:42.938460 1661 raft_consensus.cc:348] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:42.938906 1661 raft_consensus.cc:374] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:19:42.939002 1661 raft_consensus.cc:729] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b0929c4ce2433eaef580e795381680, State: Initialized, Role: FOLLOWER
I20260811 12:19:42.939703 1661 consensus_queue.cc:260] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:42.940048 1659 ts_tablet_manager.cc:1428] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 12:19:42.940785 1658 raft_consensus.cc:348] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:42.941229 1658 raft_consensus.cc:374] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:19:42.941339 1658 raft_consensus.cc:729] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed3d2b7646e41729a509523090e6f55, State: Initialized, Role: FOLLOWER
I20260811 12:19:42.941845 1661 ts_tablet_manager.cc:1428] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 12:19:42.942497 1658 consensus_queue.cc:260] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:42.944235 1632 heartbeater.cc:503] Master 127.7.39.62:40833 was elected leader, sending a full tablet report...
I20260811 12:19:42.944919 1658 ts_tablet_manager.cc:1428] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260811 12:19:43.036091 1633 tablet.cc:2369] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:19:43.051168 1407 tablet.cc:2369] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:19:43.066488 1670 raft_consensus.cc:482] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:19:43.066795 1670 raft_consensus.cc:504] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:43.068639 1670 leader_election.cc:288] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405), aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135)
I20260811 12:19:43.070092 1667 raft_consensus.cc:482] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:19:43.070444 1667 raft_consensus.cc:504] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:43.072479 1667 leader_election.cc:288] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405), 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605)
I20260811 12:19:43.076260 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.068973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.7.39.3:58807)
0811 12:19:43.069359 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.069384 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:43.069540 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:43.069945 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:43.069951 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:43.069981 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:43.070247 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:43.070260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.071669 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.071676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:43.073597 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:43.073613 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.075401 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.075408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:43.075617 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.075668 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 12:19:43.075757 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:19:43.075910 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"spinlock_wait_cycles":23808}
I20260811 12:19:43.076957 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.069022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:58807 (local address 127.7.39.1:38405)
0811 12:19:43.069275 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:43.069282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:43.069484 (+ 202us) server_negotiation.cc:407] Connection header received
0811 12:19:43.069710 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:43.069715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:43.069802 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:43.069909 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:43.070430 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.071508 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.073801 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.075052 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.075186 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.075302 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:43.076509 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0811 12:19:43.076694 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:19:43.077394 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.069103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:49015 (local address 127.7.39.2:33135)
0811 12:19:43.069362 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:43.069367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:43.069950 (+ 583us) server_negotiation.cc:407] Connection header received
0811 12:19:43.070231 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:43.070236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:43.070306 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:43.070391 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:43.071026 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.071909 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.074412 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.075826 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.075945 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.076050 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:43.076901 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 12:19:43.077071 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260811 12:19:43.078150 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.069010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33135 (local address 127.7.39.3:49015)
0811 12:19:43.069853 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.069870 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:43.070079 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:43.070522 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:43.070525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:43.070543 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:43.070875 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:43.070887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.072075 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.072081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:43.074203 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:43.074221 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.076272 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.076281 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:43.076688 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.076726 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:19:43.077594 (+ 868us) client_negotiation.cc:240] Negotiation successful
0811 12:19:43.077774 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":438,"threads_started":1}
I20260811 12:19:43.078047 1095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f962e23b884d97b20600a9ef5a80ad" candidate_uuid: "4ed3d2b7646e41729a509523090e6f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b0929c4ce2433eaef580e795381680" is_pre_election: true
I20260811 12:19:43.078756 1095 raft_consensus.cc:2454] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed3d2b7646e41729a509523090e6f55 in term 0.
I20260811 12:19:43.079665 1438 leader_election.cc:302] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed3d2b7646e41729a509523090e6f55, 57b0929c4ce2433eaef580e795381680; no voters:
I20260811 12:19:43.079674 1331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f962e23b884d97b20600a9ef5a80ad" candidate_uuid: "4ed3d2b7646e41729a509523090e6f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aab77284fd3b4b65926810ae8e50e95e" is_pre_election: true
I20260811 12:19:43.080173 1670 raft_consensus.cc:2781] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260811 12:19:43.080245 1170 tablet.cc:2369] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:19:43.080242 1331 raft_consensus.cc:2454] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ed3d2b7646e41729a509523090e6f55 in term 0.
I20260811 12:19:43.080317 1670 raft_consensus.cc:482] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:19:43.080408 1670 raft_consensus.cc:3037] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:19:43.083518 1670 raft_consensus.cc:504] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:43.084767 1095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f962e23b884d97b20600a9ef5a80ad" candidate_uuid: "4ed3d2b7646e41729a509523090e6f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b0929c4ce2433eaef580e795381680"
I20260811 12:19:43.084935 1331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f962e23b884d97b20600a9ef5a80ad" candidate_uuid: "4ed3d2b7646e41729a509523090e6f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aab77284fd3b4b65926810ae8e50e95e"
I20260811 12:19:43.085011 1095 raft_consensus.cc:3037] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:19:43.085196 1331 raft_consensus.cc:3037] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:19:43.086571 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.073410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:45195 (local address 127.7.39.1:38405)
0811 12:19:43.077223 (+ 3813us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:43.077229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:43.079131 (+ 1902us) server_negotiation.cc:407] Connection header received
0811 12:19:43.079419 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:43.079424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:43.079504 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:43.079565 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:43.080444 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.081648 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.083875 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.085123 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.085368 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.085481 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:43.086124 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 12:19:43.086269 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3691,"thread_start_us":4449,"threads_started":1}
I20260811 12:19:43.087469 1673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.072795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.7.39.2:45195)
0811 12:19:43.079023 (+ 6228us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.079044 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:43.079247 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:43.079807 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:43.079812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:43.079841 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:43.080250 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:43.080264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.081782 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.081791 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:43.083724 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:43.083739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.085724 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.085732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:43.085923 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.085959 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:19:43.086962 (+ 1003us) client_negotiation.cc:240] Negotiation successful
0811 12:19:43.087109 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6075,"thread_start_us":8644,"threads_started":1}
I20260811 12:19:43.087839 1670 leader_election.cc:288] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [CANDIDATE]: Term 1 election: Requested vote from peers 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405), aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135)
I20260811 12:19:43.088361 1095 raft_consensus.cc:2454] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed3d2b7646e41729a509523090e6f55 in term 1.
I20260811 12:19:43.088449 1331 raft_consensus.cc:2454] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ed3d2b7646e41729a509523090e6f55 in term 1.
I20260811 12:19:43.088387 1094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f962e23b884d97b20600a9ef5a80ad" candidate_uuid: "aab77284fd3b4b65926810ae8e50e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b0929c4ce2433eaef580e795381680" is_pre_election: true
I20260811 12:19:43.088838 1094 raft_consensus.cc:2379] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aab77284fd3b4b65926810ae8e50e95e in current term 1: Already voted for candidate 4ed3d2b7646e41729a509523090e6f55 in this term.
I20260811 12:19:43.088881 1438 leader_election.cc:302] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed3d2b7646e41729a509523090e6f55, 57b0929c4ce2433eaef580e795381680; no voters:
I20260811 12:19:43.089521 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.072936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33605 (local address 127.7.39.2:45191)
0811 12:19:43.081805 (+ 8869us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.081840 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:19:43.081985 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:19:43.082334 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:19:43.082338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:19:43.082364 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:19:43.082827 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:43.082839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.084143 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.084148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:43.086254 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:19:43.086268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.088361 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.088370 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:19:43.088718 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.088752 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:19:43.088830 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 12:19:43.089086 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8742,"thread_start_us":5716,"threads_started":1}
I20260811 12:19:43.089681 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.073330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:45191 (local address 127.7.39.3:33605)
0811 12:19:43.073812 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:43.073817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:43.081951 (+ 8134us) server_negotiation.cc:407] Connection header received
0811 12:19:43.082120 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:43.082125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:43.082200 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:43.082293 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:19:43.083002 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.084019 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.086435 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.087594 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.087687 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.089210 (+ 1523us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:43.089262 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 12:19:43.089391 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337}
I20260811 12:19:43.089496 1670 raft_consensus.cc:2781] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:19:43.090353 1670 raft_consensus.cc:686] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 1 LEADER]: Becoming Leader. State: Replica: 4ed3d2b7646e41729a509523090e6f55, State: Running, Role: LEADER
I20260811 12:19:43.090641 1550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f962e23b884d97b20600a9ef5a80ad" candidate_uuid: "aab77284fd3b4b65926810ae8e50e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed3d2b7646e41729a509523090e6f55" is_pre_election: true
I20260811 12:19:43.090924 1670 consensus_queue.cc:237] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:19:43.091648 1207 leader_election.cc:302] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aab77284fd3b4b65926810ae8e50e95e; no voters: 4ed3d2b7646e41729a509523090e6f55, 57b0929c4ce2433eaef580e795381680
I20260811 12:19:43.092100 1667 raft_consensus.cc:2726] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 12:19:43.095656 868 catalog_manager.cc:5495] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3). New cstate: current_term: 1 leader_uuid: "4ed3d2b7646e41729a509523090e6f55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } health_report { overall_health: HEALTHY } } }
I20260811 12:19:43.124436 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.118778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46268 (local address 127.7.39.3:33605)
0811 12:19:43.119013 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:43.119019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:43.119130 (+ 111us) server_negotiation.cc:407] Connection header received
0811 12:19:43.119383 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:43.119387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:43.119489 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:43.119630 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:19:43.120034 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.120828 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.122283 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.122677 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.122788 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.123292 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:19:43.123802 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:19:43.123939 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:43.124043 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 12:19:43.124135 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:19:43.132833 1095 raft_consensus.cc:1261] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Refusing update from remote peer 4ed3d2b7646e41729a509523090e6f55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:19:43.132838 1331 raft_consensus.cc:1261] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Refusing update from remote peer 4ed3d2b7646e41729a509523090e6f55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:19:43.133775 1677 consensus_queue.cc:1035] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:19:43.134140 1670 consensus_queue.cc:1035] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:19:43.147398 1682 mvcc.cc:205] Tried to move back new op lower bound from 7317302407702429696 to 7317302407544991744. Current Snapshot: MvccSnapshot[applied={T|T < 7317302407702429696 or (T in {7317302407702429696})}]
I20260811 12:19:43.154161 1687 mvcc.cc:205] Tried to move back new op lower bound from 7317302407702429696 to 7317302407544991744. Current Snapshot: MvccSnapshot[applied={T|T < 7317302407702429696}]
I20260811 12:19:43.159279 869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:19:43.159616 869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:19:43.163707 869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Sending DeleteTablet for 3 replicas of tablet 17f962e23b884d97b20600a9ef5a80ad
I20260811 12:19:43.164814 1297 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f962e23b884d97b20600a9ef5a80ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:19:43 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:54816
I20260811 12:19:43.164922 1068 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f962e23b884d97b20600a9ef5a80ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:19:43 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:50706
I20260811 12:19:43.165925 1296 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f962e23b884d97b20600a9ef5a80ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:19:43 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:54816
I20260811 12:19:43.166077 1516 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f962e23b884d97b20600a9ef5a80ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:19:43 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:46258
I20260811 12:19:43.166250 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 921
I20260811 12:19:43.167214 1700 tablet_replica.cc:330] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: stopping tablet replica
I20260811 12:19:43.167701 1700 raft_consensus.cc:2229] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:19:43.168224 1700 raft_consensus.cc:2258] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:19:43.169020 1704 tablet_replica.cc:330] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: stopping tablet replica
I20260811 12:19:43.169502 1704 raft_consensus.cc:2229] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:19:43.170156 1704 raft_consensus.cc:2258] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:19:43.170873 1702 tablet_replica.cc:330] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: stopping tablet replica
I20260811 12:19:43.171252 1702 raft_consensus.cc:2229] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:19:43.171767 1702 raft_consensus.cc:2258] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:19:43.173130 1704 ts_tablet_manager.cc:1905] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:19:43.173861 1700 ts_tablet_manager.cc:1905] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:19:43.180689 1700 ts_tablet_manager.cc:1918] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:19:43.180868 1700 log.cc:1196] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/17f962e23b884d97b20600a9ef5a80ad
I20260811 12:19:43.180848 1704 ts_tablet_manager.cc:1918] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:19:43.181025 1704 log.cc:1196] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/17f962e23b884d97b20600a9ef5a80ad
I20260811 12:19:43.181492 1700 ts_tablet_manager.cc:1939] T 17f962e23b884d97b20600a9ef5a80ad P aab77284fd3b4b65926810ae8e50e95e: Deleting consensus metadata
I20260811 12:19:43.181614 1704 ts_tablet_manager.cc:1939] T 17f962e23b884d97b20600a9ef5a80ad P 4ed3d2b7646e41729a509523090e6f55: Deleting consensus metadata
I20260811 12:19:43.183444 852 catalog_manager.cc:4841] TS aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135): tablet 17f962e23b884d97b20600a9ef5a80ad (table test-table [id=2101f54d75a243d9a92e92d4ae26f543]) successfully deleted
I20260811 12:19:43.183612 849 catalog_manager.cc:4841] TS 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605): tablet 17f962e23b884d97b20600a9ef5a80ad (table test-table [id=2101f54d75a243d9a92e92d4ae26f543]) successfully deleted
W20260811 12:19:43.184095 852 catalog_manager.cc:4808] TS aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135): delete failed for tablet 17f962e23b884d97b20600a9ef5a80ad because the tablet was not found. No further retry: Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
I20260811 12:19:43.185751 1702 ts_tablet_manager.cc:1905] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:19:43.186151 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.167525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.7.39.1:38405)
0811 12:19:43.167754 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:43.167760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:43.167780 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:19:43.167843 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:43.167848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:43.167941 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:43.168049 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:43.169857 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.171026 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.173180 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.173775 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.173904 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.177673 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:43.177690 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:43.177709 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:43.177757 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:43.181958 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:43.182905 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:43.182915 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:43.182918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:43.183099 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:43.184344 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:43.184348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:43.184352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:43.184829 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:43.185136 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:43.185531 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 12:19:43.185799 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:19:43.186950 1160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:43.340214 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 921
I20260811 12:19:43.373631 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 1175
W20260811 12:19:43.373929 842 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.39.1:38405 (error 108)
I20260811 12:19:43.376101 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.375518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50724)
0811 12:19:43.375795 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.375917 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260811 12:19:43.376375 842 catalog_manager.cc:4584] TS 57b0929c4ce2433eaef580e795381680 (127.7.39.1:38405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17f962e23b884d97b20600a9ef5a80ad: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
I20260811 12:19:43.390069 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.374591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.7.39.2:33135)
0811 12:19:43.375253 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:43.375260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:43.375279 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:19:43.375344 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:43.375350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:43.375428 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:43.375544 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:43.376995 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.378209 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.379576 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.380237 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.380368 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.382942 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:43.382959 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:43.382983 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:43.383025 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:43.386738 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:43.387545 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:43.387555 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:43.387559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:43.387720 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:43.388137 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:43.388142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:43.388146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:43.388668 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:43.388985 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:43.389460 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 12:19:43.389706 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514}
I20260811 12:19:43.390939 1392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:43.430393 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.429830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50736)
0811 12:19:43.430097 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.430222 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 12:19:43.431644 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.431205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50738)
0811 12:19:43.431372 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.431483 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:19:43.485347 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 1175
I20260811 12:19:43.494082 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.493466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50740)
0811 12:19:43.493708 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.493831 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 12:19:43.495345 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.494925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50754)
0811 12:19:43.495100 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.495200 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:19:43.510799 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 1409
I20260811 12:19:43.527686 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.511746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46272 (local address 127.7.39.3:33605)
0811 12:19:43.511964 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:19:43.511970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:19:43.511999 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:19:43.512187 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:19:43.512193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:19:43.512298 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:19:43.512436 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:19:43.513787 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.514915 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.516338 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:19:43.517085 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:19:43.517450 (+ 365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:19:43.520278 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:19:43.520298 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:19:43.520322 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:19:43.520393 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:19:43.524561 (+ 4168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:43.525368 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:43.525379 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:43.525382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:43.525576 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:19:43.525918 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:19:43.525930 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:19:43.525933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:19:43.526425 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:19:43.526725 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 12:19:43.527036 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 12:19:43.527275 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:19:43.528472 1619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:43.576092 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.575445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50766)
0811 12:19:43.575730 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.575853 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 12:19:43.577322 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.576844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50782)
0811 12:19:43.577053 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.577182 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 12:19:43.654201 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 1409
I20260811 12:19:43.679817 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 809
I20260811 12:19:43.681016 913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:19:43.797132 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 809
I20260811 12:19:43.797206 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:19:43.796507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:38405 (local address 127.0.0.1:50798)
0811 12:19:43.796785 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 12:19:43.796893 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:20:03.831694 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40833
--webserver_interface=127.7.39.62
--webserver_port=45785
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:40833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:04.188346 1714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:04.188701 1714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:04.188845 1714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:04.196313 1714 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:20:04.196448 1714 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:20:04.196506 1714 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:20:04.196578 1714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:04.196661 1714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:04.196759 1714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:04.196822 1714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:04.196899 1714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:04.207456 1714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:40833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:40833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=45785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:04.208963 1714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:04.210510 1714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:04.223150 1730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:04.223085 1725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:04.223124 1726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:04.224115 1714 server_base.cc:1034] running on GCE node
I20260811 12:20:04.224973 1714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:04.226562 1714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:04.227826 1714 hybrid_clock.cc:648] HybridClock initialized: now 1786450804227750 us; error 49 us; skew 500 ppm
I20260811 12:20:04.228199 1714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:04 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:04.231472 1714 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:20:04.233095 1714 webserver.cc:460] Webserver started at http://127.7.39.62:45785/ using document root <none> and password file <none>
I20260811 12:20:04.233927 1714 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:04.234038 1714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:04.239967 1714 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 12:20:04.243134 1743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:04.244596 1714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:20:04.244808 1714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "3cb90125183c4b5a99b12499648bebda"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "3979d9da92e985efbf327800de55442f"
server_key_iv: "a740aa4b2b873f67c4df21426138cbf7"
server_key_version: "encryptionkey@0"
I20260811 12:20:04.245903 1714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:04.288544 1714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:04.293447 1714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:04.293769 1714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:04.312897 1714 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:40833
I20260811 12:20:04.312942 1832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:40833 every 8 connection(s)
I20260811 12:20:04.314507 1714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:04.321868 1834 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:20:04.324295 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 1714
I20260811 12:20:04.325212 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:38405
--local_ip_for_outbound_sockets=127.7.39.1
--tserver_master_addrs=127.7.39.62:40833
--webserver_port=41983
--webserver_interface=127.7.39.1
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:20:04.325464 1834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Bootstrap starting.
I20260811 12:20:04.346988 1834 log.cc:824] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:04.364172 1834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:20:04.364789 1834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Bootstrap complete.
I20260811 12:20:04.374704 1834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:20:04.375924 1834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb90125183c4b5a99b12499648bebda, State: Initialized, Role: FOLLOWER
I20260811 12:20:04.376808 1834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:20:04.377122 1834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:04.377329 1834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:04.377549 1834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:20:04.383504 1834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:20:04.384230 1834 leader_election.cc:302] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cb90125183c4b5a99b12499648bebda; no voters:
I20260811 12:20:04.385051 1834 leader_election.cc:288] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 12:20:04.385224 1838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:20:04.388177 1838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [term 3 LEADER]: Becoming Leader. State: Replica: 3cb90125183c4b5a99b12499648bebda, State: Running, Role: LEADER
I20260811 12:20:04.389032 1838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } }
I20260811 12:20:04.390578 1834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:04.398730 1839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cb90125183c4b5a99b12499648bebda. Latest consensus state: current_term: 3 leader_uuid: "3cb90125183c4b5a99b12499648bebda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } } }
I20260811 12:20:04.399200 1839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:04.398759 1840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3cb90125183c4b5a99b12499648bebda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb90125183c4b5a99b12499648bebda" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 40833 } } }
I20260811 12:20:04.399724 1840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:04.403303 1848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:04.419252 1848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:04.420413 1848 catalog_manager.cc:1259] Loaded cluster ID: add29195478a495c8454f2ea4effcf90
I20260811 12:20:04.420511 1848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:04.424835 1848 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:04.427083 1848 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Loaded TSK: 0
I20260811 12:20:04.427413 1848 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3cb90125183c4b5a99b12499648bebda: Loaded TSK: 1
F20260811 12:20:04.428838 1848 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb4289689ad google::LogMessage::Fail() at ??:0
@ 0x7fb42896b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb4289684de google::LogMessage::Flush() at ??:0
@ 0x7fb428969a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb42c6cc7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb42c6bb73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb43e39b1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb43e39a783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb43e4262a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fb43e4260c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fb43e4705e9 std::function<>::operator()() at ??:0
@ 0x7fb43e39ecdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb43e39cc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb43e424610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb43e424439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb42b5b4345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb42b5b791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb42b5b7799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb42b592eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb42e86c6db start_thread at ??:0
@ 0x7fb42780261f clone at ??:0
W20260811 12:20:04.806561 1836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:04.806892 1836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:04.806995 1836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:04.814795 1836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:04.814932 1836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:04.815060 1836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:20:04.825614 1836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:38405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=41983
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:04.826975 1836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:04.828336 1836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:04.841986 1911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:04.842082 1908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:04.841995 1907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:04.843029 1836 server_base.cc:1034] running on GCE node
I20260811 12:20:04.843657 1836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:04.844676 1836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:04.845925 1836 hybrid_clock.cc:648] HybridClock initialized: now 1786450804845858 us; error 49 us; skew 500 ppm
I20260811 12:20:04.846283 1836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:04 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:04.849965 1836 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:20:04.851758 1836 webserver.cc:460] Webserver started at http://127.7.39.1:41983/ using document root <none> and password file <none>
I20260811 12:20:04.852545 1836 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:04.852687 1836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:04.858824 1836 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:20:04.862386 1925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:04.863991 1836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:20:04.864192 1836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "57b0929c4ce2433eaef580e795381680"
format_stamp: "Formatted at 2026-08-11 12:19:18 on dist-test-slave-rmhg"
server_key: "f9a2ec521d7eb2c476b5d2209d604dbe"
server_key_iv: "42a7eb60dfbfbd3fbb164e6c481c8eb4"
server_key_version: "encryptionkey@0"
I20260811 12:20:04.865321 1836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:04.878325 1836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:04.882925 1836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:04.883217 1836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:04.884264 1836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:04.886296 1937 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 12:20:04.892590 1836 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 12:20:04.892783 1836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20260811 12:20:04.892923 1836 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 12:20:04.896467 1836 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 12:20:04.896575 1836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.000s
I20260811 12:20:04.897099 1937 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Bootstrap starting.
I20260811 12:20:04.906520 1937 log.cc:824] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:04.926075 1937 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:20:04.927065 1937 tablet_bootstrap.cc:492] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Bootstrap complete.
I20260811 12:20:04.927846 1937 ts_tablet_manager.cc:1397] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.008s
I20260811 12:20:04.935437 1836 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:38405
I20260811 12:20:04.935458 2101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:38405 every 8 connection(s)
I20260811 12:20:04.937404 1836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:20:04.939747 1937 raft_consensus.cc:348] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:20:04.940862 1937 raft_consensus.cc:729] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b0929c4ce2433eaef580e795381680, State: Initialized, Role: FOLLOWER
I20260811 12:20:04.941709 1937 consensus_queue.cc:260] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:20:04.943236 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:04.940567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:47497)
0811 12:20:04.941609 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:04.941900 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":433,"threads_started":1}
I20260811 12:20:04.943537 1937 ts_tablet_manager.cc:1428] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
W20260811 12:20:04.943988 2102 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:40833 (0 consecutive failures): Network error: Failed to ping master at 127.7.39.62:40833: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
I20260811 12:20:04.944909 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:04.944554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:50469)
0811 12:20:04.944713 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:04.944801 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:20:04.945859 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:04.945497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:40923)
0811 12:20:04.945645 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:04.945731 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 12:20:04.946187 2102 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:20:04.946503 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 1836
I20260811 12:20:04.947113 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:33135
--local_ip_for_outbound_sockets=127.7.39.2
--tserver_master_addrs=127.7.39.62:40833
--webserver_port=45583
--webserver_interface=127.7.39.2
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:05.307621 2107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:05.307962 2107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:05.308096 2107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:05.315793 2107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:05.315979 2107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:05.316145 2107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:20:05.326730 2107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:33135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=45583
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:05.328256 2107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:05.329860 2107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:05.344090 2120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:05.344018 2116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:05.344154 2118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:05.344852 2107 server_base.cc:1034] running on GCE node
I20260811 12:20:05.345609 2107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:05.346678 2107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:05.347908 2107 hybrid_clock.cc:648] HybridClock initialized: now 1786450805347833 us; error 71 us; skew 500 ppm
I20260811 12:20:05.348258 2107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:05 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:05.351805 2107 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:20:05.353549 2107 webserver.cc:460] Webserver started at http://127.7.39.2:45583/ using document root <none> and password file <none>
I20260811 12:20:05.354302 2107 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:05.354451 2107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:05.360410 2107 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:20:05.363509 2128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:05.365042 2107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:20:05.365290 2107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "aab77284fd3b4b65926810ae8e50e95e"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "4ff0b7575311fa65b83094b61acde690"
server_key_iv: "2310cc19f749d003a7dd70e7fda6ffac"
server_key_version: "encryptionkey@0"
I20260811 12:20:05.366333 2107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:05.379739 2107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:05.384410 2107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:05.384719 2107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:05.385834 2107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:05.387272 2107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:05.387374 2107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:05.387483 2107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:05.387553 2107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:05.420382 2107 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:33135
I20260811 12:20:05.420459 2321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:33135 every 8 connection(s)
I20260811 12:20:05.422042 2107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:20:05.427238 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:05.425254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:55947)
0811 12:20:05.426039 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:05.426265 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":299,"threads_started":1}
W20260811 12:20:05.427940 2322 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:40833 (0 consecutive failures): Network error: Failed to ping master at 127.7.39.62:40833: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
I20260811 12:20:05.428800 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:05.428453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:47169)
0811 12:20:05.428613 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:05.428686 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 12:20:05.429502 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 2107
I20260811 12:20:05.429811 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:05.429368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:48895)
0811 12:20:05.429522 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:05.429667 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:20:05.430064 2322 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:20:05.430289 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:33605
--local_ip_for_outbound_sockets=127.7.39.3
--tserver_master_addrs=127.7.39.62:40833
--webserver_port=43599
--webserver_interface=127.7.39.3
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:05.790342 2325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:05.790689 2325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:05.790841 2325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:05.798285 2325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:05.798437 2325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:05.798605 2325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:20:05.808904 2325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:33605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=43599
--tserver_master_addrs=127.7.39.62:40833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:05.810447 2325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:05.811893 2325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:05.826385 2335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:05.826431 2336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:05.826449 2338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:05.827189 2325 server_base.cc:1034] running on GCE node
I20260811 12:20:05.827827 2325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:05.828835 2325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:05.830053 2325 hybrid_clock.cc:648] HybridClock initialized: now 1786450805829972 us; error 75 us; skew 500 ppm
I20260811 12:20:05.830389 2325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:05 dist-test-slave-rmhg krb5kdc[32347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450805, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:05.833806 2325 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:20:05.835428 2325 webserver.cc:460] Webserver started at http://127.7.39.3:43599/ using document root <none> and password file <none>
I20260811 12:20:05.836202 2325 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:05.836339 2325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:05.842278 2325 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 12:20:05.845440 2352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:05.846972 2325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:20:05.847165 2325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "4ed3d2b7646e41729a509523090e6f55"
format_stamp: "Formatted at 2026-08-11 12:19:19 on dist-test-slave-rmhg"
server_key: "61bda7038642eeb03a4f1967446143e9"
server_key_iv: "411659e00ca3bc57799d519ac322c008"
server_key_version: "encryptionkey@0"
I20260811 12:20:05.848296 2325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:05.873819 2325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:05.878332 2325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:05.878584 2325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:05.879683 2325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:05.881127 2325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:05.881260 2325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:05.881347 2325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:05.881403 2325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:05.913657 2325 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:33605
I20260811 12:20:05.913743 2535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:33605 every 8 connection(s)
I20260811 12:20:05.915213 2325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:20:05.920463 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:05.918457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:33585)
0811 12:20:05.919278 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:05.919505 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":313,"threads_started":1}
W20260811 12:20:05.921216 2538 heartbeater.cc:644] Failed to heartbeat to 127.7.39.62:40833 (0 consecutive failures): Network error: Failed to ping master at 127.7.39.62:40833: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
I20260811 12:20:05.922065 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:05.921729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:41095)
0811 12:20:05.921872 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:05.921947 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 12:20:05.922924 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 2325
I20260811 12:20:05.922952 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:05.922623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:57861)
0811 12:20:05.922763 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:05.922840 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:20:05.923179 2538 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:20:05.947610 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:05.946773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:53813)
0811 12:20:05.947284 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:05.947399 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":247,"threads_started":1}
I20260811 12:20:06.288370 2548 raft_consensus.cc:482] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:06.288687 2548 raft_consensus.cc:504] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:20:06.290583 2548 leader_election.cc:288] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135), 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605)
Aug 11 12:20:06 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
Aug 11 12:20:06 dist-test-slave-rmhg krb5kdc[32347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:20:06.315613 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:06.291079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:33605 (local address 127.7.39.1:33243)
0811 12:20:06.293761 (+ 2682us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:06.293782 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:06.293934 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:06.296296 (+ 2362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:06.296299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:06.296935 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:06.297821 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:06.297847 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:06.299668 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:06.299674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:06.301402 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:06.301419 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:06.302457 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:06.302462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:06.302668 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:06.303386 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:06.303418 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:06.306435 (+ 3017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:06.311316 (+ 4881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:06.311328 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:06.311347 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:06.312048 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:06.312607 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:06.312612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:06.312616 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:06.312837 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:06.313822 (+ 985us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:06.313831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:06.314253 (+ 422us) client_negotiation.cc:769] Sending connection context
0811 12:20:06.314607 (+ 354us) client_negotiation.cc:240] Negotiation successful
0811 12:20:06.315188 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2531,"mutex_wait_us":153,"spinlock_wait_cycles":24192,"thread_start_us":2304,"threads_started":1}
I20260811 12:20:06.317181 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:06.291443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:33243 (local address 127.7.39.3:33605)
0811 12:20:06.293448 (+ 2005us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:06.293464 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:06.293895 (+ 431us) server_negotiation.cc:407] Connection header received
0811 12:20:06.295516 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:06.295526 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:06.295969 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:06.296320 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:06.298037 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:06.299439 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:06.301565 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:06.302282 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:06.302442 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:06.306931 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:06.306945 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:06.306970 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:06.307027 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:06.311072 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:06.312258 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:06.312268 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:06.312278 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:06.312472 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:06.312974 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:06.312980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:06.312982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:06.313668 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:06.313862 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:06.316177 (+ 2315us) server_negotiation.cc:299] Negotiation successful
0811 12:20:06.316718 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1794,"thread_start_us":1586,"threads_started":1}
I20260811 12:20:06.320525 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:06.290931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33135 (local address 127.7.39.1:50945)
0811 12:20:06.291311 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:06.291355 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:06.292071 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:06.295953 (+ 3882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:06.295966 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:06.296935 (+ 969us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:06.297823 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:06.297847 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:06.299670 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:06.299675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:06.301409 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:06.301424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:06.303729 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:06.303736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:06.303934 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:06.304575 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:06.304598 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:06.309485 (+ 4887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:06.316610 (+ 7125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:06.316616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:06.316621 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:06.317267 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:06.317963 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:06.317968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:06.317971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:06.318135 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:06.319425 (+ 1290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:06.319430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:06.319641 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 12:20:06.319810 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 12:20:06.320151 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":179,"spinlock_wait_cycles":1920}
I20260811 12:20:06.321115 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:06.291175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:50945 (local address 127.7.39.2:33135)
0811 12:20:06.292964 (+ 1789us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:06.292986 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:06.293047 (+ 61us) server_negotiation.cc:407] Connection header received
0811 12:20:06.294963 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:06.294974 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:06.295426 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:06.295729 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:06.298063 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:06.299397 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:06.302671 (+ 3274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:06.303511 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:06.303695 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:06.309893 (+ 6198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:06.309908 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:06.309934 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:06.309994 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:06.316413 (+ 6419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:06.317488 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:06.317499 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:06.317510 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:06.317812 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:06.318279 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:06.318284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:06.318287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:06.319262 (+ 975us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:06.319502 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:06.320197 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 12:20:06.320736 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1561,"thread_start_us":1405,"threads_started":1}
W20260811 12:20:06.322131 1931 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
W20260811 12:20:06.324980 1932 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
I20260811 12:20:06.325207 1932 leader_election.cc:302] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57b0929c4ce2433eaef580e795381680; no voters: 4ed3d2b7646e41729a509523090e6f55, aab77284fd3b4b65926810ae8e50e95e
I20260811 12:20:06.325736 2548 raft_consensus.cc:2726] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:20:06.431599 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:06.430657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:41365)
0811 12:20:06.431212 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:06.431346 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":285,"threads_started":1}
I20260811 12:20:06.924630 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:06.923814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:58537)
0811 12:20:06.924282 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:06.924399 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":220,"threads_started":1}
I20260811 12:20:06.949313 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:06.948584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:33737)
0811 12:20:06.949018 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:06.949110 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260811 12:20:07.433462 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:07.432637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:51357)
0811 12:20:07.433179 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:07.433289 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":257,"threads_started":1}
I20260811 12:20:07.926355 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:07.925606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:41863)
0811 12:20:07.926069 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:07.926167 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":232,"threads_started":1}
I20260811 12:20:07.951047 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:07.950225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:49893)
0811 12:20:07.950742 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:07.950832 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":247,"threads_started":1}
I20260811 12:20:07.986078 2562 raft_consensus.cc:482] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:07.986270 2562 raft_consensus.cc:504] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:20:07.986955 2562 leader_election.cc:288] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135), 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605)
W20260811 12:20:07.988001 1931 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
W20260811 12:20:07.988199 1932 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
I20260811 12:20:07.988333 1932 leader_election.cc:302] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57b0929c4ce2433eaef580e795381680; no voters: 4ed3d2b7646e41729a509523090e6f55, aab77284fd3b4b65926810ae8e50e95e
I20260811 12:20:07.988622 2562 raft_consensus.cc:2726] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:20:08.435343 2563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:08.434489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:58919)
0811 12:20:08.435034 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:08.435146 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":235,"threads_started":1}
I20260811 12:20:08.927943 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:08.927230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:42641)
0811 12:20:08.927683 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:08.927771 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":224,"threads_started":1}
I20260811 12:20:08.952808 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:08.952022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:45729)
0811 12:20:08.952510 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:08.952600 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":263,"threads_started":1}
I20260811 12:20:09.437206 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:09.436323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:56383)
0811 12:20:09.436869 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:09.436979 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":294,"threads_started":1}
I20260811 12:20:09.929994 2573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:09.928928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:37665)
0811 12:20:09.929695 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:09.929800 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":430,"threads_started":1}
I20260811 12:20:09.954887 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:09.954045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:43645)
0811 12:20:09.954545 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:09.954694 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":225,"threads_started":1}
I20260811 12:20:10.439102 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:10.438312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:33067)
0811 12:20:10.438835 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:10.438931 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":289,"threads_started":1}
I20260811 12:20:10.931777 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:10.930993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:36595)
0811 12:20:10.931489 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:10.931590 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":266,"threads_started":1}
I20260811 12:20:10.956610 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:10.955808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:35687)
0811 12:20:10.956343 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:10.956432 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":307,"threads_started":1}
I20260811 12:20:11.198321 2579 raft_consensus.cc:482] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:11.198522 2579 raft_consensus.cc:504] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:20:11.199397 2579 leader_election.cc:288] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135), 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605)
W20260811 12:20:11.200495 1931 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
W20260811 12:20:11.200736 1932 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
I20260811 12:20:11.200865 1932 leader_election.cc:302] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57b0929c4ce2433eaef580e795381680; no voters: 4ed3d2b7646e41729a509523090e6f55, aab77284fd3b4b65926810ae8e50e95e
I20260811 12:20:11.201130 2579 raft_consensus.cc:2726] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:20:11.440845 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:11.440075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:38105)
0811 12:20:11.440567 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:11.440657 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":259,"threads_started":1}
I20260811 12:20:11.933704 2581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:11.932809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:54585)
0811 12:20:11.933413 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:11.933507 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":404,"threads_started":1}
I20260811 12:20:11.958464 2582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:11.957637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:54989)
0811 12:20:11.958154 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:11.958249 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":273,"threads_started":1}
I20260811 12:20:12.442647 2583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:12.441854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:49259)
0811 12:20:12.442364 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:12.442452 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":220,"threads_started":1}
I20260811 12:20:12.935730 2584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:12.934752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:37261)
0811 12:20:12.935316 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:12.935432 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":257,"threads_started":1}
I20260811 12:20:12.960206 2586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:12.959418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:53697)
0811 12:20:12.959928 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:12.960041 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":247,"threads_started":1}
I20260811 12:20:13.444478 2587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:13.443698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:50317)
0811 12:20:13.444179 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:13.444269 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":242,"threads_started":1}
I20260811 12:20:13.937898 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:13.936874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:55041)
0811 12:20:13.937492 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:13.937608 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":327,"threads_started":1}
I20260811 12:20:13.961900 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:13.961079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:58909)
0811 12:20:13.961625 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:13.961714 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":283,"threads_started":1}
I20260811 12:20:14.168411 2592 raft_consensus.cc:482] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:14.168666 2592 raft_consensus.cc:504] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:20:14.169450 2592 leader_election.cc:288] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135), 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605)
W20260811 12:20:14.170614 1931 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
W20260811 12:20:14.170822 1932 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
I20260811 12:20:14.170920 1932 leader_election.cc:302] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57b0929c4ce2433eaef580e795381680; no voters: 4ed3d2b7646e41729a509523090e6f55, aab77284fd3b4b65926810ae8e50e95e
I20260811 12:20:14.171159 2592 raft_consensus.cc:2726] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:20:14.446496 2596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:14.445705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:54795)
0811 12:20:14.446228 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:14.446325 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":241,"threads_started":1}
I20260811 12:20:14.939733 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:14.938939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:45097)
0811 12:20:14.939469 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:14.939574 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":260,"threads_started":1}
I20260811 12:20:14.963666 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:14.962910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:51543)
0811 12:20:14.963389 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:14.963485 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":257,"threads_started":1}
I20260811 12:20:15.448233 2603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:15.447506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:42909)
0811 12:20:15.447944 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:15.448034 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
I20260811 12:20:15.941576 2604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:15.940737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:51619)
0811 12:20:15.941253 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:15.941350 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":247,"threads_started":1}
I20260811 12:20:15.965365 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:15.964576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:42709)
0811 12:20:15.965042 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:15.965134 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":226,"threads_started":1}
I20260811 12:20:16.449966 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:16.449146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:52363)
0811 12:20:16.449701 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:16.449793 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":251,"threads_started":1}
I20260811 12:20:16.943320 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:16.942536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:60901)
0811 12:20:16.943071 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:16.943158 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":256,"threads_started":1}
I20260811 12:20:16.967115 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:16.966373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:48513)
0811 12:20:16.966853 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:16.966942 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":259,"threads_started":1}
I20260811 12:20:17.451687 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:17.450932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:57953)
0811 12:20:17.451414 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:17.451497 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":248,"threads_started":1}
I20260811 12:20:17.945230 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:17.944329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:43981)
0811 12:20:17.944876 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:17.944971 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":301,"threads_started":1}
I20260811 12:20:17.968930 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:17.968088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:40757)
0811 12:20:17.968603 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:17.968700 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":248,"threads_started":1}
I20260811 12:20:18.453576 2618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:18.452709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:55889)
0811 12:20:18.453315 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:18.453400 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":324,"threads_started":1}
I20260811 12:20:18.946985 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:18.946225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:56825)
0811 12:20:18.946722 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:18.946814 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":244,"threads_started":1}
I20260811 12:20:18.970819 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:18.969976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:55449)
0811 12:20:18.970496 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:18.970626 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":259,"threads_started":1}
I20260811 12:20:19.014868 2623 raft_consensus.cc:482] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:19.015105 2623 raft_consensus.cc:504] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b0929c4ce2433eaef580e795381680" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 38405 } } peers { permanent_uuid: "aab77284fd3b4b65926810ae8e50e95e" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33135 } } peers { permanent_uuid: "4ed3d2b7646e41729a509523090e6f55" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 33605 } }
I20260811 12:20:19.015856 2623 leader_election.cc:288] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135), 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605)
W20260811 12:20:19.016889 1932 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aab77284fd3b4b65926810ae8e50e95e (127.7.39.2:33135): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
W20260811 12:20:19.017093 1931 leader_election.cc:341] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ed3d2b7646e41729a509523090e6f55 (127.7.39.3:33605): Not found: Tablet not found: 17f962e23b884d97b20600a9ef5a80ad
I20260811 12:20:19.017210 1931 leader_election.cc:302] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57b0929c4ce2433eaef580e795381680; no voters: 4ed3d2b7646e41729a509523090e6f55, aab77284fd3b4b65926810ae8e50e95e
I20260811 12:20:19.017452 2623 raft_consensus.cc:2726] T 17f962e23b884d97b20600a9ef5a80ad P 57b0929c4ce2433eaef580e795381680 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:20:19.455437 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:19.454572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:55555)
0811 12:20:19.455128 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:19.455214 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":298,"threads_started":1}
I20260811 12:20:19.948799 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:19.948000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:49303)
0811 12:20:19.948508 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:19.948598 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":250,"threads_started":1}
I20260811 12:20:19.972821 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:19.972036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:52091)
0811 12:20:19.972550 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:19.972640 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":276,"threads_started":1}
I20260811 12:20:20.457716 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:20.456796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.2:34183)
0811 12:20:20.457422 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:20.457534 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":332,"threads_started":1}
I20260811 12:20:20.926817 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 1836
I20260811 12:20:20.944916 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:20.928103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.7.39.1:38405)
0811 12:20:20.928657 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:20.928666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:20.928701 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:20:20.928797 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:20.928806 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:20.929062 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:20.929249 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:20.931041 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:20.932081 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:20.933645 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:20.934289 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:20.934408 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:20.937495 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:20.937517 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:20.937542 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:20.937591 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:20.941867 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:20.942744 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:20.942752 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:20.942754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:20.942913 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:20.943305 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:20.943308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:20.943310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:20.943725 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:20.943981 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:20.944343 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 12:20:20.944597 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":244,"threads_started":1}
I20260811 12:20:20.946336 2093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:20.951280 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:20.950093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.3:53281)
0811 12:20:20.950997 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:20.951093 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":636,"threads_started":1}
I20260811 12:20:21.038165 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:21.037183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:40833 (local address 127.7.39.1:57355)
0811 12:20:21.037761 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:21.037877 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.62:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":290,"threads_started":1}
I20260811 12:20:21.038194 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 1836
I20260811 12:20:21.067147 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 2107
I20260811 12:20:21.081599 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:21.068053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.7.39.2:33135)
0811 12:20:21.068559 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:21.068564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:21.068582 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:20:21.068640 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:21.068644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:21.068706 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:21.068803 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:21.070313 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:21.071323 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:21.072682 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:21.073124 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:21.073285 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:21.075969 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:21.075975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:21.075978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:21.076018 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:21.078881 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:21.079678 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:21.079682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:21.079684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:21.079831 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:21.080231 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:21.080234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:21.080236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:21.080502 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:21.080749 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:21.080985 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 12:20:21.081207 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":266,"threads_started":1}
I20260811 12:20:21.082464 2314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:21.169498 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 2107
I20260811 12:20:21.198786 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 2325
I20260811 12:20:21.213497 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:21.199656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.7.39.3:33605)
0811 12:20:21.200227 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:21.200231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:21.200250 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:20:21.200308 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:21.200312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:21.200400 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:21.200508 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:21.202211 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:21.203050 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:21.204372 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:21.204858 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:21.204958 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:21.207863 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:21.207868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:21.207872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:21.207912 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:21.210919 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:21.211660 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:21.211664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:21.211666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:21.211814 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:21.212141 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:21.212145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:21.212147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:21.212426 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:21.212648 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:21.212961 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 12:20:21.213147 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":289,"threads_started":1}
I20260811 12:20:21.214314 2527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:21.300279 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 2325
2026-08-11T12:20:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65229 ms)
[----------] 30 tests from SecurityITest (312685 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:21 dist-test-slave-rmhg krb5kdc[2650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:21 dist-test-slave-rmhg krb5kdc[2650](info): set up 2 sockets
Aug 11 12:20:21 dist-test-slave-rmhg krb5kdc[2650](info): commencing operation
krb5kdc: starting...
W20260811 12:20:22.572095 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:22 dist-test-slave-rmhg krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:20:22.905067 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45711
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:35397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:45711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:23.262953 2671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:23.263283 2671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:23.263378 2671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:23.271057 2671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:23.271193 2671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:23.271241 2671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:23.271299 2671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:23.271342 2671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:23.281582 2671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:45711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:23.282995 2671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:23.284390 2671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:23.297782 2679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:23.297689 2682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:23.297739 2680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:23.298823 2671 server_base.cc:1034] running on GCE node
I20260811 12:20:23.299780 2671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:23.301441 2671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:23.302702 2671 hybrid_clock.cc:648] HybridClock initialized: now 1786450823302642 us; error 55 us; skew 500 ppm
I20260811 12:20:23.303107 2671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:23 dist-test-slave-rmhg krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450823, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:23.306813 2671 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:20:23.308620 2671 webserver.cc:460] Webserver started at http://127.7.39.62:39745/ using document root <none> and password file <none>
I20260811 12:20:23.309460 2671 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:23.309646 2671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:23.309991 2671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:23.313498 2671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "3d6139a849a14a19a530a0fac2be543c"
format_stamp: "Formatted at 2026-08-11 12:20:23 on dist-test-slave-rmhg"
server_key: "73ea9fdcff518f195d038659354fd759"
server_key_iv: "ba77b504f66f785a087fb6a3df2615a5"
server_key_version: "encryptionkey@0"
I20260811 12:20:23.314320 2671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "3d6139a849a14a19a530a0fac2be543c"
format_stamp: "Formatted at 2026-08-11 12:20:23 on dist-test-slave-rmhg"
server_key: "73ea9fdcff518f195d038659354fd759"
server_key_iv: "ba77b504f66f785a087fb6a3df2615a5"
server_key_version: "encryptionkey@0"
I20260811 12:20:23.320425 2671 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 12:20:23.324617 2698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:23.326117 2671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:20:23.326314 2671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "3d6139a849a14a19a530a0fac2be543c"
format_stamp: "Formatted at 2026-08-11 12:20:23 on dist-test-slave-rmhg"
server_key: "73ea9fdcff518f195d038659354fd759"
server_key_iv: "ba77b504f66f785a087fb6a3df2615a5"
server_key_version: "encryptionkey@0"
I20260811 12:20:23.326525 2671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:23.348409 2671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:23.353457 2671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:23.353760 2671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:23.373612 2671 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:45711
I20260811 12:20:23.373613 2787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:45711 every 8 connection(s)
I20260811 12:20:23.375231 2671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:23.378233 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 2671
I20260811 12:20:23.378504 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:23.379344 7324 external_mini_cluster.cc:1428] Setting key 59c0b5f6d57ba5337729ac731f65fd73
I20260811 12:20:23.380360 2788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:23.394301 2788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c: Bootstrap starting.
Aug 11 12:20:23 dist-test-slave-rmhg krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:23.398156 2788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:23.399518 2788 log.cc:824] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:23.403072 2788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c: No bootstrap required, opened a new log
I20260811 12:20:23.405808 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:23.381895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.7.39.62:45711)
0811 12:20:23.383020 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:23.383039 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:23.383087 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:20:23.385845 (+ 2758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:23.385876 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:23.386419 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:23.386885 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:23.388622 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:23.390054 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:23.391584 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:23.392374 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:23.392583 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:23.397293 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:23.397304 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:23.397322 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:23.397370 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:23.401140 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:23.401996 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:23.402004 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:23.402012 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:23.402206 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:23.402629 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:23.402633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:23.402635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:23.403240 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:23.403582 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:23.404079 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 12:20:23.404586 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":487,"threads_started":1}
I20260811 12:20:23.411887 2788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6139a849a14a19a530a0fac2be543c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45711 } }
I20260811 12:20:23.412329 2788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:23.412442 2788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6139a849a14a19a530a0fac2be543c, State: Initialized, Role: FOLLOWER
I20260811 12:20:23.413048 2788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6139a849a14a19a530a0fac2be543c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45711 } }
I20260811 12:20:23.413375 2788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:23.413532 2788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:23.413710 2788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:23.416076 2788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6139a849a14a19a530a0fac2be543c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45711 } }
I20260811 12:20:23.416682 2788 leader_election.cc:302] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d6139a849a14a19a530a0fac2be543c; no voters:
I20260811 12:20:23.417415 2788 leader_election.cc:288] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:23.417568 2801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:23.418917 2801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [term 1 LEADER]: Becoming Leader. State: Replica: 3d6139a849a14a19a530a0fac2be543c, State: Running, Role: LEADER
I20260811 12:20:23.419616 2801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6139a849a14a19a530a0fac2be543c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45711 } }
I20260811 12:20:23.419914 2788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:23.422580 2802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d6139a849a14a19a530a0fac2be543c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6139a849a14a19a530a0fac2be543c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45711 } } }
I20260811 12:20:23.422572 2803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d6139a849a14a19a530a0fac2be543c. Latest consensus state: current_term: 1 leader_uuid: "3d6139a849a14a19a530a0fac2be543c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d6139a849a14a19a530a0fac2be543c" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45711 } } }
I20260811 12:20:23.423076 2803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:23.423076 2802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:23.424081 2810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:23.430696 2810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:23.440263 2810 catalog_manager.cc:1347] Generated new cluster ID: fa2481febc394e6c835a0560faed170a
I20260811 12:20:23.440402 2810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:23.452626 2810 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:23.454792 2810 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:23.472638 2810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c: Generated new TSK 0
I20260811 12:20:23.473628 2810 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:20:23.564477 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45711
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:35397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:23.941964 2832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:23.942297 2832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:23.942425 2832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:23.950309 2832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:23.950453 2832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:23.950616 2832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:20:23.960907 2832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:23.962339 2832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:23.963795 2832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:23.977661 2846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:23.977619 2841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:23.977978 2837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:23.978626 2832 server_base.cc:1034] running on GCE node
I20260811 12:20:23.979233 2832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:23.980232 2832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:23.981446 2832 hybrid_clock.cc:648] HybridClock initialized: now 1786450823981383 us; error 58 us; skew 500 ppm
I20260811 12:20:23.981774 2832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:23 dist-test-slave-rmhg krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450823, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:23.985276 2832 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:20:23.986896 2832 webserver.cc:460] Webserver started at http://127.7.39.1:33601/ using document root <none> and password file <none>
I20260811 12:20:23.987649 2832 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:23.987775 2832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:23.988101 2832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:23.991250 2832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63"
format_stamp: "Formatted at 2026-08-11 12:20:23 on dist-test-slave-rmhg"
server_key: "dc331d991b12eb9689e2d125a357eba7"
server_key_iv: "e5e5e9ca41713f6a41b1711f326a417f"
server_key_version: "encryptionkey@0"
I20260811 12:20:23.992093 2832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63"
format_stamp: "Formatted at 2026-08-11 12:20:23 on dist-test-slave-rmhg"
server_key: "dc331d991b12eb9689e2d125a357eba7"
server_key_iv: "e5e5e9ca41713f6a41b1711f326a417f"
server_key_version: "encryptionkey@0"
I20260811 12:20:23.998164 2832 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 12:20:24.001798 2860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:24.003183 2832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:24.003337 2832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63"
format_stamp: "Formatted at 2026-08-11 12:20:23 on dist-test-slave-rmhg"
server_key: "dc331d991b12eb9689e2d125a357eba7"
server_key_iv: "e5e5e9ca41713f6a41b1711f326a417f"
server_key_version: "encryptionkey@0"
I20260811 12:20:24.003535 2832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:24.017532 2832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:24.022135 2832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:24.022433 2832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:24.023447 2832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:24.024860 2832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:24.024940 2832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:24.025065 2832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:24.025125 2832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:24.057050 2832 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:46087
I20260811 12:20:24.057098 3040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:46087 every 8 connection(s)
I20260811 12:20:24.058436 2832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:20:24.063560 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 2832
I20260811 12:20:24.063786 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:20:24.064705 7324 external_mini_cluster.cc:1428] Setting key f61937b33138c1bca3c8fb0f897dc18d
Aug 11 12:20:24 dist-test-slave-rmhg krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450823, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:24.081889 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:24.061246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:42221 (local address 127.7.39.62:45711)
0811 12:20:24.061850 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:24.061865 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:24.062933 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 12:20:24.065589 (+ 2656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:24.065595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:24.065694 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:24.065818 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:24.067976 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:24.068722 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:24.070962 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:24.071406 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:24.071510 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:24.075834 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:24.075839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:24.075842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:24.075881 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:24.078477 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:24.079369 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:24.079373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:24.079376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:24.079507 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:24.079963 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:24.079966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:24.079968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:24.080243 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:24.080353 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:24.081281 (+ 928us) server_negotiation.cc:299] Negotiation successful
0811 12:20:24.081532 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":269,"threads_started":1}
I20260811 12:20:24.082756 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:24.061911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45711 (local address 127.7.39.1:42221)
0811 12:20:24.062740 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:24.062791 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:24.065243 (+ 2452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:24.066151 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:24.066163 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:24.066899 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:24.067778 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:24.067808 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:24.068884 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:24.068890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:24.070812 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:24.070828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:24.071645 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:24.071652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:24.071949 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:24.072692 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:24.072721 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:24.075539 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:24.078643 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:24.078652 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:24.078670 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:24.079247 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:24.079603 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:24.079608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:24.079611 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:24.079867 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:24.080359 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:24.080367 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:24.080835 (+ 468us) client_negotiation.cc:769] Sending connection context
0811 12:20:24.081080 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 12:20:24.081550 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":310,"threads_started":1}
I20260811 12:20:24.084419 3042 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45711
I20260811 12:20:24.084745 3042 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:24.085563 3042 heartbeater.cc:511] Master 127.7.39.62:45711 requested a full tablet report, sending...
I20260811 12:20:24.087821 2731 ts_manager.cc:194] Registered new tserver with Master: bf9e630d5e834f48b4d8b95c5f1e7c63 (127.7.39.1:46087)
I20260811 12:20:24.089395 2731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:42221
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:20:24.136991 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45711
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:35397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:24.496238 3050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:24.496579 3050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:24.496727 3050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:24.504643 3050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:24.504802 3050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:24.504971 3050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:20:24.515388 3050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:24.516814 3050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:24.518319 3050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:24.532055 3057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:24.532033 3058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:24.531992 3067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:24.533406 3050 server_base.cc:1034] running on GCE node
I20260811 12:20:24.534154 3050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:24.535220 3050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:24.536466 3050 hybrid_clock.cc:648] HybridClock initialized: now 1786450824536393 us; error 58 us; skew 500 ppm
I20260811 12:20:24.536841 3050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:24 dist-test-slave-rmhg krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450824, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:24.540234 3050 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:20:24.541930 3050 webserver.cc:460] Webserver started at http://127.7.39.2:34409/ using document root <none> and password file <none>
I20260811 12:20:24.542685 3050 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:24.542797 3050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:24.543099 3050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:24.546464 3050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "16846b6a09ad48fdb55c86da264c7dc6"
format_stamp: "Formatted at 2026-08-11 12:20:24 on dist-test-slave-rmhg"
server_key: "94a3c7a38e5b142d1cacf7bf6edc6ec6"
server_key_iv: "1c754de46a25ae88fe446097c8b968f8"
server_key_version: "encryptionkey@0"
I20260811 12:20:24.547403 3050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "16846b6a09ad48fdb55c86da264c7dc6"
format_stamp: "Formatted at 2026-08-11 12:20:24 on dist-test-slave-rmhg"
server_key: "94a3c7a38e5b142d1cacf7bf6edc6ec6"
server_key_iv: "1c754de46a25ae88fe446097c8b968f8"
server_key_version: "encryptionkey@0"
I20260811 12:20:24.553352 3050 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 12:20:24.557322 3084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:24.558885 3050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:24.559056 3050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "16846b6a09ad48fdb55c86da264c7dc6"
format_stamp: "Formatted at 2026-08-11 12:20:24 on dist-test-slave-rmhg"
server_key: "94a3c7a38e5b142d1cacf7bf6edc6ec6"
server_key_iv: "1c754de46a25ae88fe446097c8b968f8"
server_key_version: "encryptionkey@0"
I20260811 12:20:24.559278 3050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:24.578524 3050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:24.584064 3050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:24.584371 3050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:24.585826 3050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:24.587309 3050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:24.587395 3050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:24.587498 3050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:24.587550 3050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:24.617120 3050 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:37147
I20260811 12:20:24.617197 3256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:37147 every 8 connection(s)
I20260811 12:20:24.618525 3050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:20:24.621047 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 3050
I20260811 12:20:24.621276 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:20:24.621898 7324 external_mini_cluster.cc:1428] Setting key be89ed89a4713e073686dd9544f644ec
Aug 11 12:20:24 dist-test-slave-rmhg krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450824, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:24.640076 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:24.621472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:47243 (local address 127.7.39.62:45711)
0811 12:20:24.621983 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:24.621991 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:24.623462 (+ 1471us) server_negotiation.cc:407] Connection header received
0811 12:20:24.625677 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:24.625682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:24.625811 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:24.625908 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:24.627530 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:24.628303 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:24.629996 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:24.630455 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:24.630567 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:24.634266 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:24.634271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:24.634274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:24.634308 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:24.636837 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:24.637672 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:24.637677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:24.637680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:24.637842 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:24.638266 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:24.638270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:24.638273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:24.638621 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:24.638773 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:24.639465 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 12:20:24.639697 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":280,"threads_started":1}
I20260811 12:20:24.640913 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:24.622380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45711 (local address 127.7.39.2:47243)
0811 12:20:24.623262 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:24.623327 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:24.625395 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:24.626182 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:24.626191 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:24.626740 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:24.627357 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:24.627376 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:24.628447 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:24.628451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:24.629844 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:24.629857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:24.630699 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:24.630707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:24.630939 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:24.631592 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:24.631624 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:24.634019 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:24.636992 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:24.637000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:24.637020 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:24.637543 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:24.637937 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:24.637963 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:24.637966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:24.638156 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:24.638735 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:24.638741 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:24.639101 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 12:20:24.639335 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 12:20:24.639763 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":333,"threads_started":1}
I20260811 12:20:24.642517 3257 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45711
I20260811 12:20:24.642864 3257 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:24.643639 3257 heartbeater.cc:511] Master 127.7.39.62:45711 requested a full tablet report, sending...
I20260811 12:20:24.645174 2731 ts_manager.cc:194] Registered new tserver with Master: 16846b6a09ad48fdb55c86da264c7dc6 (127.7.39.2:37147)
I20260811 12:20:24.646006 2731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:47243
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:20:24.691887 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45711
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:35397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:25.042637 3270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:25.042944 3270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:25.043047 3270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:25.050676 3270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:25.050815 3270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:25.050933 3270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:20:25.061379 3270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:35397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:45711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:25.062708 3270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:25.064181 3270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:25.077734 3278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:25.077920 3279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:25.078003 3285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:25.078948 3270 server_base.cc:1034] running on GCE node
I20260811 12:20:25.079582 3270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:25.080593 3270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:25.081833 3270 hybrid_clock.cc:648] HybridClock initialized: now 1786450825081777 us; error 38 us; skew 500 ppm
I20260811 12:20:25.082223 3270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:25 dist-test-slave-rmhg krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450825, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:25.085521 3270 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:20:25.087001 3270 webserver.cc:460] Webserver started at http://127.7.39.3:40555/ using document root <none> and password file <none>
I20260811 12:20:25.087682 3270 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:25.087783 3270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:25.088074 3270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:25.091096 3270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "6f6f78581060457bbf241f229159ddfb"
format_stamp: "Formatted at 2026-08-11 12:20:25 on dist-test-slave-rmhg"
server_key: "cb7a0872e6035fa126882cbd0edf1990"
server_key_iv: "319b4e7440f574f61432f8e2c551e8aa"
server_key_version: "encryptionkey@0"
I20260811 12:20:25.091868 3270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "6f6f78581060457bbf241f229159ddfb"
format_stamp: "Formatted at 2026-08-11 12:20:25 on dist-test-slave-rmhg"
server_key: "cb7a0872e6035fa126882cbd0edf1990"
server_key_iv: "319b4e7440f574f61432f8e2c551e8aa"
server_key_version: "encryptionkey@0"
I20260811 12:20:25.092535 3042 heartbeater.cc:503] Master 127.7.39.62:45711 was elected leader, sending a full tablet report...
I20260811 12:20:25.097805 3270 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 12:20:25.101847 3296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:25.103547 3270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:25.103724 3270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "6f6f78581060457bbf241f229159ddfb"
format_stamp: "Formatted at 2026-08-11 12:20:25 on dist-test-slave-rmhg"
server_key: "cb7a0872e6035fa126882cbd0edf1990"
server_key_iv: "319b4e7440f574f61432f8e2c551e8aa"
server_key_version: "encryptionkey@0"
I20260811 12:20:25.103940 3270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:25.119621 3270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:25.124229 3270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:25.124545 3270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:25.125622 3270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:25.127115 3270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:25.127202 3270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:25.127333 3270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:25.127393 3270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:25.159605 3270 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:38945
I20260811 12:20:25.159650 3499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:38945 every 8 connection(s)
I20260811 12:20:25.161031 3270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:20:25.164081 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 3270
I20260811 12:20:25.164304 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:20:25.164999 7324 external_mini_cluster.cc:1428] Setting key e1502258cc29758b0ca2069724f533ba
Aug 11 12:20:25 dist-test-slave-rmhg krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450825, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:25.183408 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.163814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:58119 (local address 127.7.39.62:45711)
0811 12:20:25.164280 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.164286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.165923 (+ 1637us) server_negotiation.cc:407] Connection header received
0811 12:20:25.168054 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.168059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.168154 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.168267 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:25.170008 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.170798 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.172428 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.172857 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.172963 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.176717 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:25.176723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:25.176726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:25.176775 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:25.179712 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.180906 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.180912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.180918 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.181083 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.181692 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.181697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.181701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.182037 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:25.182158 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.182909 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.183129 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":233,"threads_started":1}
I20260811 12:20:25.184465 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.164563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45711 (local address 127.7.39.3:58119)
0811 12:20:25.165453 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:25.165509 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:25.167726 (+ 2217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:25.168568 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:25.168581 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:25.169186 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:25.169823 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.169843 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.170913 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.170917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.172257 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.172272 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.173072 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.173078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.173339 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.174087 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:25.174112 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:25.176452 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:25.179936 (+ 3484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:25.179948 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:25.179971 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:25.180732 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:25.181257 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:25.181263 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:25.181266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:25.181547 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:25.182158 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:25.182165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:25.182547 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 12:20:25.182810 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 12:20:25.183278 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":346,"threads_started":1}
I20260811 12:20:25.186026 3500 heartbeater.cc:347] Connected to a master server at 127.7.39.62:45711
I20260811 12:20:25.186376 3500 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:25.187166 3500 heartbeater.cc:511] Master 127.7.39.62:45711 requested a full tablet report, sending...
I20260811 12:20:25.188755 2731 ts_manager.cc:194] Registered new tserver with Master: 6f6f78581060457bbf241f229159ddfb (127.7.39.3:38945)
I20260811 12:20:25.189622 2731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:58119
I20260811 12:20:25.194629 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:20:25.212935 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.199347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.7.39.62:45711)
0811 12:20:25.199577 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.199585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.199996 (+ 411us) server_negotiation.cc:407] Connection header received
0811 12:20:25.200208 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.200213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.200308 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.200406 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:25.201936 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.202691 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.204081 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.204508 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.204616 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.207241 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:25.207247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:25.207250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:25.207292 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:25.210398 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.211091 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.211095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.211097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.211220 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.211527 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.211530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.211531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.211903 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:25.212093 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.212397 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.212630 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:20:25.217541 2731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:20:25.225315 2731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:20:25.252354 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.238110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37147 (local address 127.0.0.1:49318)
0811 12:20:25.239655 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:25.239709 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:25.239892 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:25.242472 (+ 2580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:25.242481 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:25.242519 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:25.242958 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.242979 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.246548 (+ 3569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.246554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.249912 (+ 3358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.249925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.251335 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.251341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.251531 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.251586 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 12:20:25.251719 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 12:20:25.251931 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"thread_start_us":11058,"threads_started":1}
I20260811 12:20:25.252628 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.238309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.7.39.2:37147)
0811 12:20:25.240508 (+ 2199us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.240516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.240545 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:20:25.240652 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.240659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.240922 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.243206 (+ 2284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:25.243444 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.246380 (+ 2936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.250106 (+ 3726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.251134 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.251256 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.251492 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.252119 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.252307 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2053,"thread_start_us":1924,"threads_started":1}
I20260811 12:20:25.256938 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.238449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:38945 (local address 127.0.0.1:33656)
0811 12:20:25.247697 (+ 9248us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:25.247724 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:25.247847 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:25.248731 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:25.248737 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:25.248762 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:25.249598 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.249614 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.251845 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.251850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.254230 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.254245 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.256163 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.256170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.256352 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.256384 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:20:25.256471 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:20:25.256596 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9096,"thread_start_us":979,"threads_started":1}
I20260811 12:20:25.257056 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.239114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33656 (local address 127.7.39.3:38945)
0811 12:20:25.248032 (+ 8918us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.248042 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.248078 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:20:25.248186 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.248194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.248451 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.248684 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:25.249855 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.251650 (+ 1795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.254415 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.255836 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.256024 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.256324 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.256568 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.256733 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8730,"thread_start_us":8565,"threads_started":1}
I20260811 12:20:25.256994 3160 tablet_service.cc:1467] Processing CreateTablet for tablet ab704fccd6f94afcb7cf0758ddb2279d (DEFAULT_TABLE table=test-table [id=cbd6459da5904082bc165ff3d94f94a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:25.258599 3160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab704fccd6f94afcb7cf0758ddb2279d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:25.259419 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.241214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:46087 (local address 127.0.0.1:58880)
0811 12:20:25.249453 (+ 8239us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:25.249478 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:25.249612 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:25.252742 (+ 3130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:25.252747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:25.252775 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:25.253426 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.253436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.255211 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.255216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.257185 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.257201 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.258690 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.258696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.258858 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.258891 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:20:25.258972 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:20:25.259111 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8006,"thread_start_us":6131,"threads_started":1}
I20260811 12:20:25.259799 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.241592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.7.39.1:46087)
0811 12:20:25.252103 (+ 10511us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.252112 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.252143 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:20:25.252236 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.252244 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.252512 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.252952 (+ 440us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:25.253600 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.255023 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.257378 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.258464 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.258602 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.258836 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.259082 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.259258 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10326,"thread_start_us":10166,"threads_started":1}
I20260811 12:20:25.261513 3380 tablet_service.cc:1467] Processing CreateTablet for tablet ab704fccd6f94afcb7cf0758ddb2279d (DEFAULT_TABLE table=test-table [id=cbd6459da5904082bc165ff3d94f94a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:25.262881 3380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab704fccd6f94afcb7cf0758ddb2279d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:25.264371 2944 tablet_service.cc:1467] Processing CreateTablet for tablet ab704fccd6f94afcb7cf0758ddb2279d (DEFAULT_TABLE table=test-table [id=cbd6459da5904082bc165ff3d94f94a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:25.265987 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab704fccd6f94afcb7cf0758ddb2279d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:25.273952 3536 tablet_bootstrap.cc:492] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Bootstrap starting.
I20260811 12:20:25.276053 3537 tablet_bootstrap.cc:492] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Bootstrap starting.
I20260811 12:20:25.278689 3536 tablet_bootstrap.cc:654] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:25.279528 3537 tablet_bootstrap.cc:654] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:25.279774 3536 log.cc:824] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:25.280871 3537 log.cc:824] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:25.281177 3538 tablet_bootstrap.cc:492] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Bootstrap starting.
I20260811 12:20:25.282922 3536 tablet_bootstrap.cc:492] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: No bootstrap required, opened a new log
I20260811 12:20:25.283198 3536 ts_tablet_manager.cc:1397] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260811 12:20:25.283957 3537 tablet_bootstrap.cc:492] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: No bootstrap required, opened a new log
I20260811 12:20:25.284231 3537 ts_tablet_manager.cc:1397] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:20:25.285552 3538 tablet_bootstrap.cc:654] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:25.286819 3538 log.cc:824] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:25.291203 3538 tablet_bootstrap.cc:492] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: No bootstrap required, opened a new log
I20260811 12:20:25.291142 3536 raft_consensus.cc:348] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.291477 3538 ts_tablet_manager.cc:1397] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 12:20:25.291530 3536 raft_consensus.cc:374] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:25.291625 3536 raft_consensus.cc:729] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16846b6a09ad48fdb55c86da264c7dc6, State: Initialized, Role: FOLLOWER
I20260811 12:20:25.292215 3536 consensus_queue.cc:260] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.293360 3257 heartbeater.cc:503] Master 127.7.39.62:45711 was elected leader, sending a full tablet report...
I20260811 12:20:25.294011 3536 ts_tablet_manager.cc:1428] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
I20260811 12:20:25.294219 3537 raft_consensus.cc:348] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.294663 3537 raft_consensus.cc:374] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:25.294761 3537 raft_consensus.cc:729] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f6f78581060457bbf241f229159ddfb, State: Initialized, Role: FOLLOWER
I20260811 12:20:25.295377 3537 consensus_queue.cc:260] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.296556 3500 heartbeater.cc:503] Master 127.7.39.62:45711 was elected leader, sending a full tablet report...
I20260811 12:20:25.297240 3537 ts_tablet_manager.cc:1428] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260811 12:20:25.301898 3538 raft_consensus.cc:348] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.302317 3538 raft_consensus.cc:374] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:25.302423 3538 raft_consensus.cc:729] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9e630d5e834f48b4d8b95c5f1e7c63, State: Initialized, Role: FOLLOWER
I20260811 12:20:25.303018 3538 consensus_queue.cc:260] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.304592 3538 ts_tablet_manager.cc:1428] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
W20260811 12:20:25.310876 3044 tablet.cc:2369] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:25.353102 3552 raft_consensus.cc:482] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:25.353428 3552 raft_consensus.cc:504] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.355243 3552 leader_election.cc:288] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16846b6a09ad48fdb55c86da264c7dc6 (127.7.39.2:37147), 6f6f78581060457bbf241f229159ddfb (127.7.39.3:38945)
I20260811 12:20:25.362187 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.355662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:34189 (local address 127.7.39.2:37147)
0811 12:20:25.355877 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.355882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.356729 (+ 847us) server_negotiation.cc:407] Connection header received
0811 12:20:25.356920 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.356925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.356998 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.357078 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:25.357723 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.358431 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.360280 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.361116 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.361293 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.361393 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.361778 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.361909 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:20:25.362524 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.355928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:44075 (local address 127.7.39.3:38945)
0811 12:20:25.356226 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.356233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.356926 (+ 693us) server_negotiation.cc:407] Connection header received
0811 12:20:25.357096 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.357101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.357187 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.357268 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:25.357876 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.358631 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.360513 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.361560 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.361695 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.361797 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.362145 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.362277 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:20:25.362571 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.355736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:38945 (local address 127.7.39.1:44075)
0811 12:20:25.356797 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:25.356811 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:25.356951 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:25.357309 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:25.357313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:25.357343 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:25.357707 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.357716 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.358776 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.358780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.360342 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.360356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.361789 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.361795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.361951 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.361985 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:20:25.362083 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 12:20:25.362227 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"thread_start_us":707,"threads_started":1}
I20260811 12:20:25.362993 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.355681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37147 (local address 127.7.39.1:34189)
0811 12:20:25.356591 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:25.356610 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:25.356771 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:25.357108 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:25.357112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:25.357138 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:25.357555 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.357567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.358561 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.358566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.360114 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:25.360127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.361373 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.361379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:25.361569 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.361604 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:20:25.362456 (+ 852us) client_negotiation.cc:240] Negotiation successful
0811 12:20:25.362603 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":533,"threads_started":1}
I20260811 12:20:25.363552 3412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab704fccd6f94afcb7cf0758ddb2279d" candidate_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f6f78581060457bbf241f229159ddfb" is_pre_election: true
I20260811 12:20:25.364053 3185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab704fccd6f94afcb7cf0758ddb2279d" candidate_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16846b6a09ad48fdb55c86da264c7dc6" is_pre_election: true
I20260811 12:20:25.364204 3412 raft_consensus.cc:2454] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf9e630d5e834f48b4d8b95c5f1e7c63 in term 0.
I20260811 12:20:25.364620 3185 raft_consensus.cc:2454] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf9e630d5e834f48b4d8b95c5f1e7c63 in term 0.
I20260811 12:20:25.364977 2863 leader_election.cc:302] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f6f78581060457bbf241f229159ddfb, bf9e630d5e834f48b4d8b95c5f1e7c63; no voters:
I20260811 12:20:25.365487 3552 raft_consensus.cc:2781] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:20:25.365633 3552 raft_consensus.cc:482] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:20:25.365731 3552 raft_consensus.cc:3037] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:25.367985 3552 raft_consensus.cc:504] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.368578 3552 leader_election.cc:288] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [CANDIDATE]: Term 1 election: Requested vote from peers 16846b6a09ad48fdb55c86da264c7dc6 (127.7.39.2:37147), 6f6f78581060457bbf241f229159ddfb (127.7.39.3:38945)
I20260811 12:20:25.369130 3185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab704fccd6f94afcb7cf0758ddb2279d" candidate_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16846b6a09ad48fdb55c86da264c7dc6"
I20260811 12:20:25.369133 3412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab704fccd6f94afcb7cf0758ddb2279d" candidate_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f6f78581060457bbf241f229159ddfb"
I20260811 12:20:25.369370 3412 raft_consensus.cc:3037] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:25.369370 3185 raft_consensus.cc:3037] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:20:25.370573 3259 tablet.cc:2369] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:25.372615 3412 raft_consensus.cc:2454] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf9e630d5e834f48b4d8b95c5f1e7c63 in term 1.
I20260811 12:20:25.372622 3185 raft_consensus.cc:2454] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf9e630d5e834f48b4d8b95c5f1e7c63 in term 1.
I20260811 12:20:25.373067 2861 leader_election.cc:302] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16846b6a09ad48fdb55c86da264c7dc6, bf9e630d5e834f48b4d8b95c5f1e7c63; no voters:
I20260811 12:20:25.373450 3552 raft_consensus.cc:2781] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:25.373915 3552 raft_consensus.cc:686] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 1 LEADER]: Becoming Leader. State: Replica: bf9e630d5e834f48b4d8b95c5f1e7c63, State: Running, Role: LEADER
I20260811 12:20:25.374459 3552 consensus_queue.cc:237] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } }
I20260811 12:20:25.379021 2730 catalog_manager.cc:5495] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf9e630d5e834f48b4d8b95c5f1e7c63 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf9e630d5e834f48b4d8b95c5f1e7c63" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 46087 } health_report { overall_health: HEALTHY } } }
I20260811 12:20:25.405189 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.399708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58886 (local address 127.7.39.1:46087)
0811 12:20:25.399895 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.399900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.399919 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:20:25.400101 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.400104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.400223 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.400357 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:20:25.400830 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.401640 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.403085 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.403480 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.403579 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.404093 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:20:25.404662 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:20:25.404772 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.404876 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.404949 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 12:20:25.412758 3504 tablet.cc:2369] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:25.414116 3185 raft_consensus.cc:1261] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 1 FOLLOWER]: Refusing update from remote peer bf9e630d5e834f48b4d8b95c5f1e7c63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:25.414116 3412 raft_consensus.cc:1261] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 1 FOLLOWER]: Refusing update from remote peer bf9e630d5e834f48b4d8b95c5f1e7c63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:25.415201 3555 consensus_queue.cc:1035] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16846b6a09ad48fdb55c86da264c7dc6" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:25.415603 3552 consensus_queue.cc:1035] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f6f78581060457bbf241f229159ddfb" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 38945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:25.429283 3563 mvcc.cc:205] Tried to move back new op lower bound from 7317302580885024768 to 7317302580737568768. Current Snapshot: MvccSnapshot[applied={T|T < 7317302580885024768}]
I20260811 12:20:25.443897 2731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:25.444188 2731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:25.448061 2731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d6139a849a14a19a530a0fac2be543c: Sending DeleteTablet for 3 replicas of tablet ab704fccd6f94afcb7cf0758ddb2279d
I20260811 12:20:25.449383 3160 tablet_service.cc:1514] Processing DeleteTablet for tablet ab704fccd6f94afcb7cf0758ddb2279d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:25 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:49318
I20260811 12:20:25.449388 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet ab704fccd6f94afcb7cf0758ddb2279d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:25 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:58880
I20260811 12:20:25.450223 3380 tablet_service.cc:1514] Processing DeleteTablet for tablet ab704fccd6f94afcb7cf0758ddb2279d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:25 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:33656
I20260811 12:20:25.450978 3582 tablet_replica.cc:330] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: stopping tablet replica
I20260811 12:20:25.450382 3580 tablet_replica.cc:330] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: stopping tablet replica
I20260811 12:20:25.451547 3585 tablet_replica.cc:330] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: stopping tablet replica
I20260811 12:20:25.452050 3580 raft_consensus.cc:2229] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:25.452042 3585 raft_consensus.cc:2229] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:25.452626 3582 raft_consensus.cc:2229] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:20:25.452673 3585 raft_consensus.cc:2258] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:25.452736 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 2832
I20260811 12:20:25.454049 3580 raft_consensus.cc:2258] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:25.454530 3582 raft_consensus.cc:2258] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:25.457104 3580 ts_tablet_manager.cc:1905] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:25.457476 3582 ts_tablet_manager.cc:1905] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:25.457578 3585 ts_tablet_manager.cc:1905] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:25.464744 3580 ts_tablet_manager.cc:1918] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:25.465021 3585 ts_tablet_manager.cc:1918] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:25.465076 3580 log.cc:1196] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/ab704fccd6f94afcb7cf0758ddb2279d
I20260811 12:20:25.465225 3585 log.cc:1196] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/ab704fccd6f94afcb7cf0758ddb2279d
I20260811 12:20:25.465338 3582 ts_tablet_manager.cc:1918] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:25.465538 3582 log.cc:1196] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/ab704fccd6f94afcb7cf0758ddb2279d
I20260811 12:20:25.465806 3585 ts_tablet_manager.cc:1939] T ab704fccd6f94afcb7cf0758ddb2279d P 6f6f78581060457bbf241f229159ddfb: Deleting consensus metadata
I20260811 12:20:25.466109 3580 ts_tablet_manager.cc:1939] T ab704fccd6f94afcb7cf0758ddb2279d P 16846b6a09ad48fdb55c86da264c7dc6: Deleting consensus metadata
I20260811 12:20:25.466123 3582 ts_tablet_manager.cc:1939] T ab704fccd6f94afcb7cf0758ddb2279d P bf9e630d5e834f48b4d8b95c5f1e7c63: Deleting consensus metadata
I20260811 12:20:25.468029 2705 catalog_manager.cc:4841] TS 6f6f78581060457bbf241f229159ddfb (127.7.39.3:38945): tablet ab704fccd6f94afcb7cf0758ddb2279d (table test-table [id=cbd6459da5904082bc165ff3d94f94a0]) successfully deleted
I20260811 12:20:25.468111 2700 catalog_manager.cc:4841] TS 16846b6a09ad48fdb55c86da264c7dc6 (127.7.39.2:37147): tablet ab704fccd6f94afcb7cf0758ddb2279d (table test-table [id=cbd6459da5904082bc165ff3d94f94a0]) successfully deleted
I20260811 12:20:25.468075 2707 catalog_manager.cc:4841] TS bf9e630d5e834f48b4d8b95c5f1e7c63 (127.7.39.1:46087): tablet ab704fccd6f94afcb7cf0758ddb2279d (table test-table [id=cbd6459da5904082bc165ff3d94f94a0]) successfully deleted
Aug 11 12:20:25 dist-test-slave-rmhg krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:20:25.479885 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.454008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.7.39.1:46087)
0811 12:20:25.454187 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.454193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.455393 (+ 1200us) server_negotiation.cc:407] Connection header received
0811 12:20:25.455705 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.455711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.455819 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.455915 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:25.457950 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.459085 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.461170 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.461837 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.461998 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.473388 (+ 11390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:25.473403 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:25.473420 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:25.473466 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:25.476681 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.477700 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.477709 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.477711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.477882 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.478371 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.478374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.478376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.478744 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:25.479001 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.479357 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.479594 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:20:25.480613 3034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:25.580619 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 2832
I20260811 12:20:25.605103 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 3050
Aug 11 12:20:25 dist-test-slave-rmhg krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:20:25.621256 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.605935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.7.39.2:37147)
0811 12:20:25.606133 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.606139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.606160 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:20:25.606332 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.606338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.606405 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.606491 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:25.607866 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.608671 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.610047 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.610453 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.610570 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.614897 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:25.614914 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:25.614932 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:25.614978 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:25.617980 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.618790 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.618798 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.618801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.618943 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.619331 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.619335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.619337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.619845 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:25.620160 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.620561 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.620834 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:20:25.622079 3246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:25.717994 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 3050
I20260811 12:20:25.745332 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 3270
Aug 11 12:20:25 dist-test-slave-rmhg krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:20:25.761454 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:25.746212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.7.39.3:38945)
0811 12:20:25.746417 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:25.746423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:25.746446 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:20:25.746542 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:25.746548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:25.746629 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:25.746716 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:25.748176 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.748949 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.750454 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:25.750870 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:25.750992 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:25.755695 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:25.755715 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:25.755733 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:25.755778 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:25.758704 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.759485 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.759492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.759495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.759672 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:25.760037 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:25.760041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:25.760043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:25.760428 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:25.760669 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:25.760946 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 12:20:25.761182 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:20:25.762107 3492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:25.852337 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 3270
I20260811 12:20:25.879336 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 2671
I20260811 12:20:25.880585 2782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:25.993963 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 2671
2026-08-11T12:20:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4696 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T12:20:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:26Z Disabled control of system clock
I20260811 12:20:26.097470 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:43559
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:43559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:20:26.467777 3602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:26.468163 3602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:26.468300 3602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:26.477829 3602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:26.477986 3602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:26.478051 3602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:26.478142 3602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:26.478205 3602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:26.488680 3602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:43559
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:43559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:26.490387 3602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:26.492827 3602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:26.505827 3609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:26.505867 3610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:26.505888 3613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:26.506814 3602 server_base.cc:1034] running on GCE node
I20260811 12:20:26.507714 3602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:26.509383 3602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:26.510632 3602 hybrid_clock.cc:648] HybridClock initialized: now 1786450826510532 us; error 79 us; skew 500 ppm
I20260811 12:20:26.511080 3602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:26.513540 3602 webserver.cc:460] Webserver started at http://127.7.39.62:44019/ using document root <none> and password file <none>
I20260811 12:20:26.514266 3602 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:26.514362 3602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:26.514662 3602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:26.517804 3602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "6353542df44a45fbb315183e544fb502"
format_stamp: "Formatted at 2026-08-11 12:20:26 on dist-test-slave-rmhg"
server_key: "7a5dc60e8568b28255b34aec2c66d5df"
server_key_iv: "02d72c7726edb1d3e6307458209e2eab"
server_key_version: "encryptionkey@0"
I20260811 12:20:26.518790 3602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "6353542df44a45fbb315183e544fb502"
format_stamp: "Formatted at 2026-08-11 12:20:26 on dist-test-slave-rmhg"
server_key: "7a5dc60e8568b28255b34aec2c66d5df"
server_key_iv: "02d72c7726edb1d3e6307458209e2eab"
server_key_version: "encryptionkey@0"
I20260811 12:20:26.525022 3602 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 12:20:26.528952 3619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:26.530632 3602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:20:26.530810 3602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "6353542df44a45fbb315183e544fb502"
format_stamp: "Formatted at 2026-08-11 12:20:26 on dist-test-slave-rmhg"
server_key: "7a5dc60e8568b28255b34aec2c66d5df"
server_key_iv: "02d72c7726edb1d3e6307458209e2eab"
server_key_version: "encryptionkey@0"
I20260811 12:20:26.531019 3602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:26.579210 3602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:26.580236 3602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:26.580534 3602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:26.600513 3602 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:43559
I20260811 12:20:26.600533 3706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:43559 every 8 connection(s)
I20260811 12:20:26.601959 3602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:26.605814 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:26.610643 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 3602
I20260811 12:20:26.610822 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:26.611292 7324 external_mini_cluster.cc:1428] Setting key 5077ec24af4298a87f9960c6064cfff5
I20260811 12:20:26.617226 3709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502: Bootstrap starting.
I20260811 12:20:26.620541 3709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:26.621748 3709 log.cc:824] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:26.625138 3709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502: No bootstrap required, opened a new log
I20260811 12:20:26.625584 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:26.613302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.7.39.62:43559)
0811 12:20:26.614389 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:26.614401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:26.614458 (+ 57us) server_negotiation.cc:407] Connection header received
0811 12:20:26.616252 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:26.616277 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:26.616766 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:26.617186 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:26.617769 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:26.619127 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:26.620579 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:26.621444 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:26.621636 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:26.622941 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:26.622954 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:26.622980 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:26.623628 (+ 648us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:26.623713 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:26.623737 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:26.623831 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:26.624102 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 12:20:26.624511 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":386,"threads_started":1}
I20260811 12:20:26.633683 3709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353542df44a45fbb315183e544fb502" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43559 } }
I20260811 12:20:26.634207 3709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:26.634321 3709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6353542df44a45fbb315183e544fb502, State: Initialized, Role: FOLLOWER
I20260811 12:20:26.634935 3709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353542df44a45fbb315183e544fb502" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43559 } }
I20260811 12:20:26.635197 3709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:26.635299 3709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:26.635432 3709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:26.637799 3709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353542df44a45fbb315183e544fb502" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43559 } }
I20260811 12:20:26.638293 3709 leader_election.cc:302] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6353542df44a45fbb315183e544fb502; no voters:
I20260811 12:20:26.638844 3709 leader_election.cc:288] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:26.639011 3718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:26.639709 3718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [term 1 LEADER]: Becoming Leader. State: Replica: 6353542df44a45fbb315183e544fb502, State: Running, Role: LEADER
I20260811 12:20:26.640348 3718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353542df44a45fbb315183e544fb502" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43559 } }
I20260811 12:20:26.641021 3709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:26.643532 3719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6353542df44a45fbb315183e544fb502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353542df44a45fbb315183e544fb502" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43559 } } }
I20260811 12:20:26.644021 3719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:26.643923 3722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6353542df44a45fbb315183e544fb502. Latest consensus state: current_term: 1 leader_uuid: "6353542df44a45fbb315183e544fb502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353542df44a45fbb315183e544fb502" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 43559 } } }
I20260811 12:20:26.644258 3722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:26.646772 3732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:26.652222 3732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:26.663327 3732 catalog_manager.cc:1347] Generated new cluster ID: 31b56e75d2ed40a2a3a0dc53b62dfde6
I20260811 12:20:26.663465 3732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:26.681571 3732 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:26.682868 3732 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:26.695628 3732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502: Generated new TSK 0
I20260811 12:20:26.696651 3732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:26.707604 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43559
--builtin_ntp_servers=127.7.39.20:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:20:27.062620 3750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:27.062906 3750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:27.063020 3750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:27.070361 3750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:27.070478 3750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:27.070683 3750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:20:27.080039 3750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:27.081418 3750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:27.083284 3750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:27.097141 3757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:27.097205 3759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:27.097091 3761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:27.098039 3750 server_base.cc:1034] running on GCE node
I20260811 12:20:27.098716 3750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:27.099793 3750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:27.101068 3750 hybrid_clock.cc:648] HybridClock initialized: now 1786450827100991 us; error 50 us; skew 500 ppm
I20260811 12:20:27.101452 3750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:27.104130 3750 webserver.cc:460] Webserver started at http://127.7.39.1:40131/ using document root <none> and password file <none>
I20260811 12:20:27.104951 3750 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:27.105068 3750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:27.105454 3750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:27.108552 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "553006335c2347aab29307c742f24068"
format_stamp: "Formatted at 2026-08-11 12:20:27 on dist-test-slave-rmhg"
server_key: "2be8019abd820cec27b667e44d4fbc4c"
server_key_iv: "195c807556cfcb1373085754f46e63a8"
server_key_version: "encryptionkey@0"
I20260811 12:20:27.109504 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "553006335c2347aab29307c742f24068"
format_stamp: "Formatted at 2026-08-11 12:20:27 on dist-test-slave-rmhg"
server_key: "2be8019abd820cec27b667e44d4fbc4c"
server_key_iv: "195c807556cfcb1373085754f46e63a8"
server_key_version: "encryptionkey@0"
I20260811 12:20:27.115268 3750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:20:27.119091 3771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:27.120599 3750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:20:27.120779 3750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "553006335c2347aab29307c742f24068"
format_stamp: "Formatted at 2026-08-11 12:20:27 on dist-test-slave-rmhg"
server_key: "2be8019abd820cec27b667e44d4fbc4c"
server_key_iv: "195c807556cfcb1373085754f46e63a8"
server_key_version: "encryptionkey@0"
I20260811 12:20:27.121001 3750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:27.139681 3750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:27.140625 3750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:27.140929 3750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:27.142215 3750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:27.143978 3750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:27.144093 3750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:27.144203 3750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:27.144275 3750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:27.176074 3750 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:44513
I20260811 12:20:27.176131 3952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:44513 every 8 connection(s)
I20260811 12:20:27.177496 3750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:20:27.182299 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 3750
I20260811 12:20:27.182483 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:20:27.182919 7324 external_mini_cluster.cc:1428] Setting key 01c22bb097a826c60d9c4dce67659666
I20260811 12:20:27.185979 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43559
--builtin_ntp_servers=127.7.39.20:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:20:27.191002 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:27.180309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:40949 (local address 127.7.39.62:43559)
0811 12:20:27.180776 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:27.180782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:27.182194 (+ 1412us) server_negotiation.cc:407] Connection header received
0811 12:20:27.184202 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:27.184208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:27.184303 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:27.184399 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:27.185757 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:27.186561 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:27.188188 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:27.188592 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:27.188705 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:27.190145 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:27.190150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:27.190154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:27.190173 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:27.190214 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:27.190227 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:27.190300 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:27.190543 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 12:20:27.190645 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
I20260811 12:20:27.192087 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:27.180929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43559 (local address 127.7.39.1:40949)
0811 12:20:27.181999 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:27.182041 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:27.183909 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:27.184713 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:27.184726 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:27.184860 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:27.185580 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:27.185601 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:27.186720 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:27.186725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:27.188037 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:27.188050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:27.188823 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:27.188831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:27.189089 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:27.189812 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:27.189815 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:27.189868 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:27.189951 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:27.190313 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:27.190327 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:27.190335 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:20:27.190503 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 12:20:27.190933 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":352,"threads_started":1}
I20260811 12:20:27.193689 3953 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43559
I20260811 12:20:27.194008 3953 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:27.194829 3953 heartbeater.cc:511] Master 127.7.39.62:43559 requested a full tablet report, sending...
I20260811 12:20:27.196925 3650 ts_manager.cc:194] Registered new tserver with Master: 553006335c2347aab29307c742f24068 (127.7.39.1:44513)
I20260811 12:20:27.198589 3650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.1:40949
W20260811 12:20:27.535084 3960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:27.535442 3960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:27.535583 3960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:27.543291 3960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:27.543433 3960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:27.543581 3960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:20:27.553401 3960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:27.554894 3960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:27.556915 3960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:27.571604 3970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:27.571635 3967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:27.571564 3972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:27.572553 3960 server_base.cc:1034] running on GCE node
I20260811 12:20:27.573238 3960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:27.574309 3960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:27.575582 3960 hybrid_clock.cc:648] HybridClock initialized: now 1786450827575526 us; error 44 us; skew 500 ppm
I20260811 12:20:27.575910 3960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:27.578400 3960 webserver.cc:460] Webserver started at http://127.7.39.2:41977/ using document root <none> and password file <none>
I20260811 12:20:27.579190 3960 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:27.579291 3960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:27.579635 3960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:27.582978 3960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "deab955e0bea43d6ae0109d197460f71"
format_stamp: "Formatted at 2026-08-11 12:20:27 on dist-test-slave-rmhg"
server_key: "c76539e03b92f3ba5100755d60194d28"
server_key_iv: "ea7d672e3124f9be29f21eff5c2869e8"
server_key_version: "encryptionkey@0"
I20260811 12:20:27.583845 3960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "deab955e0bea43d6ae0109d197460f71"
format_stamp: "Formatted at 2026-08-11 12:20:27 on dist-test-slave-rmhg"
server_key: "c76539e03b92f3ba5100755d60194d28"
server_key_iv: "ea7d672e3124f9be29f21eff5c2869e8"
server_key_version: "encryptionkey@0"
I20260811 12:20:27.589968 3960 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 12:20:27.594480 3979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:27.596400 3960 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:20:27.596575 3960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "deab955e0bea43d6ae0109d197460f71"
format_stamp: "Formatted at 2026-08-11 12:20:27 on dist-test-slave-rmhg"
server_key: "c76539e03b92f3ba5100755d60194d28"
server_key_iv: "ea7d672e3124f9be29f21eff5c2869e8"
server_key_version: "encryptionkey@0"
I20260811 12:20:27.596786 3960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:27.622171 3960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:27.623318 3960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:27.623677 3960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:27.624795 3960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:27.626530 3960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:27.626624 3960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:27.626734 3960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:27.626799 3960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:27.659611 3960 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:33151
I20260811 12:20:27.659667 4155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:33151 every 8 connection(s)
I20260811 12:20:27.661176 3960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:20:27.668828 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 3960
I20260811 12:20:27.668994 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:20:27.669508 7324 external_mini_cluster.cc:1428] Setting key ed4f13ca11b8d9907b2a5f774a336702
I20260811 12:20:27.672695 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43559
--builtin_ntp_servers=127.7.39.20:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:20:27.674805 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:27.664079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:50725 (local address 127.7.39.62:43559)
0811 12:20:27.664327 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:27.664334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:27.665945 (+ 1611us) server_negotiation.cc:407] Connection header received
0811 12:20:27.667772 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:27.667777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:27.667850 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:27.667945 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:27.669123 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:27.669940 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:27.671673 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:27.672084 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:27.672182 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:27.674035 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:27.674041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:27.674044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:27.674066 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:27.674104 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:27.674117 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:27.674194 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:27.674461 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 12:20:27.674564 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 12:20:27.676312 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:27.664693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43559 (local address 127.7.39.2:50725)
0811 12:20:27.665770 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:27.665811 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:27.667491 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:27.668164 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:27.668173 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:27.668267 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:27.668904 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:27.668927 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:27.670125 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:27.670130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:27.671481 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:27.671506 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:27.672343 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:27.672351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:27.672719 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:27.673555 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:27.673559 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:27.673621 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:27.673744 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:27.674217 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:27.674229 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:27.674238 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:20:27.674441 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 12:20:27.674918 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":360,"threads_started":1}
I20260811 12:20:27.677937 4157 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43559
I20260811 12:20:27.678313 4157 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:27.679148 4157 heartbeater.cc:511] Master 127.7.39.62:43559 requested a full tablet report, sending...
I20260811 12:20:27.680753 3650 ts_manager.cc:194] Registered new tserver with Master: deab955e0bea43d6ae0109d197460f71 (127.7.39.2:33151)
I20260811 12:20:27.681643 3650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.2:50725
W20260811 12:20:28.020533 4160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:28.020872 4160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:28.020999 4160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:28.028651 4160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:28.028784 4160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:28.028930 4160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:20:28.038579 4160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:43559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:28.039978 4160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:28.042065 4160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:28.056049 4167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:28.056116 4172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:28.056154 4170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:28.056820 4160 server_base.cc:1034] running on GCE node
I20260811 12:20:28.057518 4160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:28.058530 4160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:28.059744 4160 hybrid_clock.cc:648] HybridClock initialized: now 1786450828059693 us; error 41 us; skew 500 ppm
I20260811 12:20:28.060062 4160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:28.062587 4160 webserver.cc:460] Webserver started at http://127.7.39.3:40861/ using document root <none> and password file <none>
I20260811 12:20:28.063366 4160 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:28.063465 4160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:28.063814 4160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:28.067041 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "2c3f226dc20541f1be16d24b828347e6"
format_stamp: "Formatted at 2026-08-11 12:20:28 on dist-test-slave-rmhg"
server_key: "13841ff6f75e9beb3d7db55b7e39764d"
server_key_iv: "c1efb36e0a0e177f37c98908a500ec69"
server_key_version: "encryptionkey@0"
I20260811 12:20:28.067930 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "2c3f226dc20541f1be16d24b828347e6"
format_stamp: "Formatted at 2026-08-11 12:20:28 on dist-test-slave-rmhg"
server_key: "13841ff6f75e9beb3d7db55b7e39764d"
server_key_iv: "c1efb36e0a0e177f37c98908a500ec69"
server_key_version: "encryptionkey@0"
I20260811 12:20:28.073591 4160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 12:20:28.077544 4179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:28.079192 4160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:28.079363 4160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "2c3f226dc20541f1be16d24b828347e6"
format_stamp: "Formatted at 2026-08-11 12:20:28 on dist-test-slave-rmhg"
server_key: "13841ff6f75e9beb3d7db55b7e39764d"
server_key_iv: "c1efb36e0a0e177f37c98908a500ec69"
server_key_version: "encryptionkey@0"
I20260811 12:20:28.079592 4160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:28.099223 4160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:28.100176 4160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:28.100479 4160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:28.101661 4160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:28.103423 4160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:28.103518 4160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:28.103615 4160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:28.103681 4160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:28.135007 4160 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:32817
I20260811 12:20:28.135075 4365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:32817 every 8 connection(s)
I20260811 12:20:28.136525 4160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:20:28.146034 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 4160
I20260811 12:20:28.146239 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:20:28.146729 7324 external_mini_cluster.cc:1428] Setting key 39ae35dcdd74b1c117579f7154135c67
I20260811 12:20:28.150558 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.139483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:41339 (local address 127.7.39.62:43559)
0811 12:20:28.139714 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.139721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.141390 (+ 1669us) server_negotiation.cc:407] Connection header received
0811 12:20:28.143345 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.143351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.143442 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.143538 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:28.144750 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.145558 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.147367 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.147854 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.147978 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.149662 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:28.149669 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:28.149673 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:28.149699 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:28.149746 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:28.149762 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:28.149886 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.150202 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.150292 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:20:28.151660 4369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.140019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:43559 (local address 127.7.39.3:41339)
0811 12:20:28.141216 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.141263 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:28.143065 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:28.143799 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:28.143808 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:28.143907 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:28.144577 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.144597 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.145733 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.145738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.147206 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.147220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.148117 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.148126 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.148435 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.149259 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:28.149262 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:28.149308 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:28.149408 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:28.149938 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:28.149952 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:28.149962 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:20:28.150151 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 12:20:28.150532 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":323,"threads_started":1}
I20260811 12:20:28.153102 4366 heartbeater.cc:347] Connected to a master server at 127.7.39.62:43559
I20260811 12:20:28.153491 4366 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:28.154292 4366 heartbeater.cc:511] Master 127.7.39.62:43559 requested a full tablet report, sending...
I20260811 12:20:28.156026 3650 ts_manager.cc:194] Registered new tserver with Master: 2c3f226dc20541f1be16d24b828347e6 (127.7.39.3:32817)
I20260811 12:20:28.156922 3650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.3:41339
I20260811 12:20:28.163120 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:20:28.174201 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.167792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.7.39.62:43559)
0811 12:20:28.168051 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.168058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.168445 (+ 387us) server_negotiation.cc:407] Connection header received
0811 12:20:28.168651 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.168655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.168739 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.168833 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:28.169495 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.170214 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.171694 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.172160 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.172284 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.173481 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:28.173486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:28.173490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:28.173512 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:28.173557 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:28.173570 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:28.173652 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.173798 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.173902 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:20:28.178624 3650 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:20:28.186055 3650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:20:28.201597 3953 heartbeater.cc:503] Master 127.7.39.62:43559 was elected leader, sending a full tablet report...
I20260811 12:20:28.211548 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.200854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35284 (local address 127.7.39.3:32817)
0811 12:20:28.204572 (+ 3718us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.204582 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.204620 (+ 38us) server_negotiation.cc:407] Connection header received
0811 12:20:28.204714 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.204722 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.204942 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.205837 (+ 895us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:28.206125 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.207149 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.209182 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.210253 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.210379 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.210683 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.211000 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.211188 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3481,"thread_start_us":3315,"threads_started":1}
I20260811 12:20:28.211591 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.199546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:32817 (local address 127.0.0.1:35284)
0811 12:20:28.200901 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.200918 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:28.201025 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:28.205184 (+ 4159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:28.205190 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:28.205216 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:28.205629 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.205643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.207296 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.207300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.208921 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.208935 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.210452 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.210459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.210774 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.210818 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 12:20:28.210930 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 12:20:28.211143 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1161,"mutex_wait_us":56,"thread_start_us":1019,"threads_started":1}
I20260811 12:20:28.211684 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.199135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:33151 (local address 127.0.0.1:42968)
0811 12:20:28.200015 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.200093 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:28.200292 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:28.203356 (+ 3064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:28.203361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:28.203395 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:28.203825 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.203837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.205893 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.205899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.208613 (+ 2714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.208630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.210662 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.210669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.210867 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.210959 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 12:20:28.211032 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:20:28.211253 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":249,"thread_start_us":393,"threads_started":1}
I20260811 12:20:28.211707 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.200975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36426)
0811 12:20:28.201505 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.201522 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:28.201628 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:28.202988 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:28.202996 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:28.203024 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:28.203490 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.203505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.205756 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.205761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.208613 (+ 2852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.208630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.210643 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.210650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.210851 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.210954 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 12:20:28.211040 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:20:28.211253 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":282,"thread_start_us":726,"threads_started":1}
I20260811 12:20:28.212967 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.199421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.7.39.2:33151)
0811 12:20:28.202689 (+ 3268us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.202699 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.202730 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:20:28.202839 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.202847 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.203114 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.204051 (+ 937us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:28.204296 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.205765 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.208875 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.210384 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.212043 (+ 1659us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.212319 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.212397 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.212587 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3082,"thread_start_us":3068,"threads_started":1}
I20260811 12:20:28.213207 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.201621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.7.39.1:44513)
0811 12:20:28.202325 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.202334 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.202364 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:20:28.202460 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.202467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.202755 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.203723 (+ 968us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:28.203967 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.205592 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.208875 (+ 3283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.210384 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.212178 (+ 1794us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.212590 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.212659 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.212832 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":396,"threads_started":1}
I20260811 12:20:28.216212 4048 tablet_service.cc:1467] Processing CreateTablet for tablet 86965dd48e3a4629ae92db66b1b91f03 (DEFAULT_TABLE table=test-table [id=241538acc8a64bffbac43f7d1e563364]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:28.216280 4260 tablet_service.cc:1467] Processing CreateTablet for tablet 86965dd48e3a4629ae92db66b1b91f03 (DEFAULT_TABLE table=test-table [id=241538acc8a64bffbac43f7d1e563364]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:28.216540 3846 tablet_service.cc:1467] Processing CreateTablet for tablet 86965dd48e3a4629ae92db66b1b91f03 (DEFAULT_TABLE table=test-table [id=241538acc8a64bffbac43f7d1e563364]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:28.217643 4048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86965dd48e3a4629ae92db66b1b91f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:28.217859 3846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86965dd48e3a4629ae92db66b1b91f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:28.217857 4260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86965dd48e3a4629ae92db66b1b91f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:28.231154 4398 tablet_bootstrap.cc:492] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: Bootstrap starting.
I20260811 12:20:28.233585 4401 tablet_bootstrap.cc:492] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068: Bootstrap starting.
I20260811 12:20:28.235503 4398 tablet_bootstrap.cc:654] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:28.237406 4398 log.cc:824] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:28.237571 4401 tablet_bootstrap.cc:654] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:28.238916 4401 log.cc:824] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:28.239403 4399 tablet_bootstrap.cc:492] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: Bootstrap starting.
I20260811 12:20:28.241704 4398 tablet_bootstrap.cc:492] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: No bootstrap required, opened a new log
I20260811 12:20:28.242022 4398 ts_tablet_manager.cc:1397] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 12:20:28.242133 4401 tablet_bootstrap.cc:492] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068: No bootstrap required, opened a new log
I20260811 12:20:28.242398 4401 ts_tablet_manager.cc:1397] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:20:28.248893 4399 tablet_bootstrap.cc:654] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:28.250247 4399 log.cc:824] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:28.253062 4399 tablet_bootstrap.cc:492] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: No bootstrap required, opened a new log
I20260811 12:20:28.252877 4398 raft_consensus.cc:348] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.252879 4401 raft_consensus.cc:348] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.253386 4399 ts_tablet_manager.cc:1397] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.005s
I20260811 12:20:28.253398 4398 raft_consensus.cc:374] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:28.253392 4401 raft_consensus.cc:374] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:28.253491 4398 raft_consensus.cc:729] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deab955e0bea43d6ae0109d197460f71, State: Initialized, Role: FOLLOWER
I20260811 12:20:28.253491 4401 raft_consensus.cc:729] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 553006335c2347aab29307c742f24068, State: Initialized, Role: FOLLOWER
I20260811 12:20:28.254158 4401 consensus_queue.cc:260] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.254163 4398 consensus_queue.cc:260] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.255779 4157 heartbeater.cc:503] Master 127.7.39.62:43559 was elected leader, sending a full tablet report...
I20260811 12:20:28.256043 4401 ts_tablet_manager.cc:1428] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 12:20:28.256439 4398 ts_tablet_manager.cc:1428] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
I20260811 12:20:28.262672 4407 raft_consensus.cc:482] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:28.263047 4407 raft_consensus.cc:504] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.263324 4399 raft_consensus.cc:348] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.263859 4399 raft_consensus.cc:374] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:28.263981 4399 raft_consensus.cc:729] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3f226dc20541f1be16d24b828347e6, State: Initialized, Role: FOLLOWER
I20260811 12:20:28.264722 4399 consensus_queue.cc:260] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.265360 4407 leader_election.cc:288] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c3f226dc20541f1be16d24b828347e6 (127.7.39.3:32817), 553006335c2347aab29307c742f24068 (127.7.39.1:44513)
I20260811 12:20:28.266113 4366 heartbeater.cc:503] Master 127.7.39.62:43559 was elected leader, sending a full tablet report...
I20260811 12:20:28.266793 4399 ts_tablet_manager.cc:1428] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260811 12:20:28.273516 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.266306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:41827 (local address 127.7.39.3:32817)
0811 12:20:28.266493 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.266498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.266520 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:20:28.266725 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.266731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.266810 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.266879 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:28.267843 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.268820 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.271182 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.272354 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.272453 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.272562 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.273105 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.273278 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:20:28.273591 4411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.265722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:32817 (local address 127.7.39.2:41827)
0811 12:20:28.266379 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.266400 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:28.266540 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:28.267138 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:28.267142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:28.267169 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:28.267661 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.267675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.269007 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.269011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.270940 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.270956 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.272749 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.272756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.272934 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.272968 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:20:28.273062 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:20:28.273245 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":912,"threads_started":1}
I20260811 12:20:28.274108 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.267120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.7.39.2:34185)
0811 12:20:28.267784 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.267805 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:28.267923 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:28.268366 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:28.268369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:28.268390 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:28.268689 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.268696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.269996 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.270001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.271805 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:28.271821 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.273492 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.273499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:28.273702 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.273735 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:20:28.273810 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 12:20:28.273897 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":801,"threads_started":1}
I20260811 12:20:28.274211 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.267655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:34185 (local address 127.7.39.1:44513)
0811 12:20:28.267849 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.267854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.267872 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:20:28.268095 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.268101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.268175 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.268223 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:28.268842 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.269794 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.272023 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.273274 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.273392 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.273481 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.273862 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.273987 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:20:28.274904 4291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86965dd48e3a4629ae92db66b1b91f03" candidate_uuid: "deab955e0bea43d6ae0109d197460f71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f226dc20541f1be16d24b828347e6" is_pre_election: true
I20260811 12:20:28.275431 3875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86965dd48e3a4629ae92db66b1b91f03" candidate_uuid: "deab955e0bea43d6ae0109d197460f71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "553006335c2347aab29307c742f24068" is_pre_election: true
I20260811 12:20:28.275573 4291 raft_consensus.cc:2454] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate deab955e0bea43d6ae0109d197460f71 in term 0.
I20260811 12:20:28.276010 3875 raft_consensus.cc:2454] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate deab955e0bea43d6ae0109d197460f71 in term 0.
I20260811 12:20:28.276293 3982 leader_election.cc:302] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3f226dc20541f1be16d24b828347e6, deab955e0bea43d6ae0109d197460f71; no voters:
I20260811 12:20:28.276724 4407 raft_consensus.cc:2781] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:20:28.276881 4407 raft_consensus.cc:482] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:20:28.276993 4407 raft_consensus.cc:3037] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:28.279325 4407 raft_consensus.cc:504] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.279932 4407 leader_election.cc:288] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [CANDIDATE]: Term 1 election: Requested vote from peers 2c3f226dc20541f1be16d24b828347e6 (127.7.39.3:32817), 553006335c2347aab29307c742f24068 (127.7.39.1:44513)
I20260811 12:20:28.280598 4291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86965dd48e3a4629ae92db66b1b91f03" candidate_uuid: "deab955e0bea43d6ae0109d197460f71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f226dc20541f1be16d24b828347e6"
I20260811 12:20:28.280606 3875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86965dd48e3a4629ae92db66b1b91f03" candidate_uuid: "deab955e0bea43d6ae0109d197460f71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "553006335c2347aab29307c742f24068"
I20260811 12:20:28.280843 3875 raft_consensus.cc:3037] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:28.280841 4291 raft_consensus.cc:3037] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:28.284231 4291 raft_consensus.cc:2454] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate deab955e0bea43d6ae0109d197460f71 in term 1.
I20260811 12:20:28.284231 3875 raft_consensus.cc:2454] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate deab955e0bea43d6ae0109d197460f71 in term 1.
I20260811 12:20:28.284754 3982 leader_election.cc:302] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3f226dc20541f1be16d24b828347e6, deab955e0bea43d6ae0109d197460f71; no voters:
I20260811 12:20:28.285099 4407 raft_consensus.cc:2781] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:28.285629 4407 raft_consensus.cc:686] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 1 LEADER]: Becoming Leader. State: Replica: deab955e0bea43d6ae0109d197460f71, State: Running, Role: LEADER
I20260811 12:20:28.286257 4407 consensus_queue.cc:237] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } }
I20260811 12:20:28.291790 3650 catalog_manager.cc:5495] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 reported cstate change: term changed from 0 to 1, leader changed from <none> to deab955e0bea43d6ae0109d197460f71 (127.7.39.2). New cstate: current_term: 1 leader_uuid: "deab955e0bea43d6ae0109d197460f71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deab955e0bea43d6ae0109d197460f71" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 33151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 } health_report { overall_health: UNKNOWN } } }
I20260811 12:20:28.320780 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.315030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42978 (local address 127.7.39.2:33151)
0811 12:20:28.315270 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.315275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.315295 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:20:28.315393 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.315397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.315494 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.315628 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:20:28.316032 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.316768 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.318372 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.318809 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.318908 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.319457 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:20:28.320027 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:20:28.320159 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.320343 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.320490 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:20:28.330158 3875 raft_consensus.cc:1261] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 1 FOLLOWER]: Refusing update from remote peer deab955e0bea43d6ae0109d197460f71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:28.330158 4291 raft_consensus.cc:1261] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 1 FOLLOWER]: Refusing update from remote peer deab955e0bea43d6ae0109d197460f71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:28.331350 4413 consensus_queue.cc:1035] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3f226dc20541f1be16d24b828347e6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:28.331758 4407 consensus_queue.cc:1035] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "553006335c2347aab29307c742f24068" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 44513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:28.345371 4419 mvcc.cc:205] Tried to move back new op lower bound from 7317302592828362752 to 7317302592665800704. Current Snapshot: MvccSnapshot[applied={T|T < 7317302592828362752}]
I20260811 12:20:28.358327 3650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:28.358650 3650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:28.362733 3650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6353542df44a45fbb315183e544fb502: Sending DeleteTablet for 3 replicas of tablet 86965dd48e3a4629ae92db66b1b91f03
I20260811 12:20:28.364123 4048 tablet_service.cc:1514] Processing DeleteTablet for tablet 86965dd48e3a4629ae92db66b1b91f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:28 UTC) from {username='slave'} at 127.0.0.1:42968
I20260811 12:20:28.364126 4260 tablet_service.cc:1514] Processing DeleteTablet for tablet 86965dd48e3a4629ae92db66b1b91f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:28 UTC) from {username='slave'} at 127.0.0.1:35284
I20260811 12:20:28.364509 3846 tablet_service.cc:1514] Processing DeleteTablet for tablet 86965dd48e3a4629ae92db66b1b91f03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:28 UTC) from {username='slave'} at 127.0.0.1:36426
I20260811 12:20:28.366535 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 3750
I20260811 12:20:28.377503 4433 tablet_replica.cc:330] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: stopping tablet replica
I20260811 12:20:28.378253 4439 tablet_replica.cc:330] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068: stopping tablet replica
I20260811 12:20:28.379065 4439 raft_consensus.cc:2229] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:28.385833 4433 raft_consensus.cc:2229] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:20:28.386693 4439 raft_consensus.cc:2258] T 86965dd48e3a4629ae92db66b1b91f03 P 553006335c2347aab29307c742f24068 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:28.387637 4433 raft_consensus.cc:2258] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:28.387305 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.367308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36442 (local address 127.7.39.1:44513)
0811 12:20:28.367499 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.367505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.377765 (+ 10260us) server_negotiation.cc:407] Connection header received
0811 12:20:28.378135 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.378141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.378283 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.378370 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:28.379182 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.380349 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.382551 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.383247 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.383362 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.385392 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:28.385409 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:28.385430 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:28.386291 (+ 861us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:28.386362 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:28.386393 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:28.386502 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.386776 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.386916 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:20:28.388453 4438 tablet_replica.cc:330] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: stopping tablet replica
I20260811 12:20:28.388859 3942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:28.389832 4438 raft_consensus.cc:2229] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:28.390892 4438 raft_consensus.cc:2258] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:28.391798 4433 ts_tablet_manager.cc:1905] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:28.393329 4438 ts_tablet_manager.cc:1905] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:28.400139 4433 ts_tablet_manager.cc:1918] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:28.400352 4433 log.cc:1196] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/86965dd48e3a4629ae92db66b1b91f03
I20260811 12:20:28.400960 4433 ts_tablet_manager.cc:1939] T 86965dd48e3a4629ae92db66b1b91f03 P deab955e0bea43d6ae0109d197460f71: Deleting consensus metadata
I20260811 12:20:28.401361 4438 ts_tablet_manager.cc:1918] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:28.401549 4438 log.cc:1196] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/86965dd48e3a4629ae92db66b1b91f03
I20260811 12:20:28.402099 4438 ts_tablet_manager.cc:1939] T 86965dd48e3a4629ae92db66b1b91f03 P 2c3f226dc20541f1be16d24b828347e6: Deleting consensus metadata
I20260811 12:20:28.403227 3625 catalog_manager.cc:4841] TS deab955e0bea43d6ae0109d197460f71 (127.7.39.2:33151): tablet 86965dd48e3a4629ae92db66b1b91f03 (table test-table [id=241538acc8a64bffbac43f7d1e563364]) successfully deleted
I20260811 12:20:28.403923 3628 catalog_manager.cc:4841] TS 2c3f226dc20541f1be16d24b828347e6 (127.7.39.3:32817): tablet 86965dd48e3a4629ae92db66b1b91f03 (table test-table [id=241538acc8a64bffbac43f7d1e563364]) successfully deleted
I20260811 12:20:28.488173 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 3750
I20260811 12:20:28.517716 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 3960
W20260811 12:20:28.517654 3628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.39.1:44513 (error 108)
I20260811 12:20:28.519516 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.519116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36446)
0811 12:20:28.519278 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.519372 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 12:20:28.519894 3628 catalog_manager.cc:4584] TS 553006335c2347aab29307c742f24068 (127.7.39.1:44513): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86965dd48e3a4629ae92db66b1b91f03: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
I20260811 12:20:28.526830 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.519176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.7.39.2:33151)
0811 12:20:28.519385 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.519391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.519410 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:20:28.519476 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.519481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.519590 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.519692 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:28.520233 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.521461 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.522853 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.523558 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.523696 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.524878 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:28.524894 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:28.524916 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:28.525862 (+ 946us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:28.525927 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:28.525964 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:28.526084 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.526308 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.526422 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:20:28.527619 4144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:28.573714 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.573235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36462)
0811 12:20:28.573446 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.573540 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 12:20:28.574822 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.574430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36474)
0811 12:20:28.574591 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.574673 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:20:28.635529 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.634958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36488)
0811 12:20:28.635195 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.635310 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 12:20:28.636580 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.636241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36492)
0811 12:20:28.636378 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.636461 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:20:28.643774 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 3960
I20260811 12:20:28.665932 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 4160
I20260811 12:20:28.672541 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.666692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.7.39.3:32817)
0811 12:20:28.666891 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:28.666895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:28.666915 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:20:28.667052 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:28.667056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:28.667136 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:28.667218 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:28.667634 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.668378 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.669775 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:28.670180 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:28.670280 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:28.671276 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:28.671286 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:28.671303 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:28.671891 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:28.671946 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:28.671967 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:28.672051 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:28.672140 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 12:20:28.672231 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:20:28.673290 4358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:28.717227 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.716592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36508)
0811 12:20:28.716846 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.716998 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 12:20:28.718434 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.717947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36512)
0811 12:20:28.718186 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.718289 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 12:20:28.763760 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 4160
I20260811 12:20:28.789400 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 3602
I20260811 12:20:28.790611 3699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:28.901626 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:28.900934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:44513 (local address 127.0.0.1:36520)
0811 12:20:28.901279 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:28.901415 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.1:44513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 12:20:28.901613 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 3602
2026-08-11T12:20:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2905 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:29 dist-test-slave-rmhg krb5kdc[4453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:29 dist-test-slave-rmhg krb5kdc[4453](info): set up 2 sockets
Aug 11 12:20:29 dist-test-slave-rmhg krb5kdc[4453](info): commencing operation
krb5kdc: starting...
W20260811 12:20:30.173890 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.009s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:30 dist-test-slave-rmhg krb5kdc[4453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:20:30.513509 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:44893
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:40115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:44893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:30.874555 4479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:30.874891 4479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:30.875006 4479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:30.882470 4479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:30.882601 4479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:30.882675 4479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:30.882759 4479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:30.882814 4479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:30.893220 4479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:40115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:44893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:44893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:30.894708 4479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:30.896148 4479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:30.909210 4484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:30.909183 4490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:30.909122 4485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:30.910862 4479 server_base.cc:1034] running on GCE node
I20260811 12:20:30.911823 4479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:30.913559 4479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:30.914816 4479 hybrid_clock.cc:648] HybridClock initialized: now 1786450830914757 us; error 51 us; skew 500 ppm
I20260811 12:20:30.915223 4479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:30 dist-test-slave-rmhg krb5kdc[4453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450830, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:30.918975 4479 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:20:30.920758 4479 webserver.cc:460] Webserver started at http://127.7.39.62:41433/ using document root <none> and password file <none>
I20260811 12:20:30.921584 4479 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:30.921705 4479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:30.922034 4479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:30.925290 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "97783fd1ef2242539b4bfadb70e017f0"
format_stamp: "Formatted at 2026-08-11 12:20:30 on dist-test-slave-rmhg"
server_key: "1f3e9a125e5998f50e73cd7a7084012b"
server_key_iv: "78af5eaf994a345c2c805cb84fa80527"
server_key_version: "encryptionkey@0"
I20260811 12:20:30.926116 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "97783fd1ef2242539b4bfadb70e017f0"
format_stamp: "Formatted at 2026-08-11 12:20:30 on dist-test-slave-rmhg"
server_key: "1f3e9a125e5998f50e73cd7a7084012b"
server_key_iv: "78af5eaf994a345c2c805cb84fa80527"
server_key_version: "encryptionkey@0"
I20260811 12:20:30.931977 4479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:20:30.936075 4504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:30.937666 4479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:20:30.937851 4479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "97783fd1ef2242539b4bfadb70e017f0"
format_stamp: "Formatted at 2026-08-11 12:20:30 on dist-test-slave-rmhg"
server_key: "1f3e9a125e5998f50e73cd7a7084012b"
server_key_iv: "78af5eaf994a345c2c805cb84fa80527"
server_key_version: "encryptionkey@0"
I20260811 12:20:30.938052 4479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:30.959220 4479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:30.963370 4479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:30.963652 4479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:30.982385 4479 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:44893
I20260811 12:20:30.982434 4582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:44893 every 8 connection(s)
I20260811 12:20:30.983852 4479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:30.987635 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 4479
I20260811 12:20:30.987843 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:30.988188 4583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:30.988485 7324 external_mini_cluster.cc:1428] Setting key 3514b0387473b2df2459e7505aae2b01
I20260811 12:20:31.004024 4583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0: Bootstrap starting.
Aug 11 12:20:31 dist-test-slave-rmhg krb5kdc[4453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:31.008199 4583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:31.009414 4583 log.cc:824] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:31.013120 4583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0: No bootstrap required, opened a new log
I20260811 12:20:31.014173 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:30.990567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46900 (local address 127.7.39.62:44893)
0811 12:20:30.991505 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:30.991518 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:30.991600 (+ 82us) server_negotiation.cc:407] Connection header received
0811 12:20:30.993454 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:30.993479 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:30.993922 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:30.994286 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:30.996189 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:30.997664 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:30.999657 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:31.000437 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:31.000621 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:31.005948 (+ 5327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:31.005961 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:31.005988 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:31.006039 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:31.009342 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:31.010452 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:31.010462 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:31.010473 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:31.010749 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:31.011231 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:31.011234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:31.011236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:31.011837 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:31.012058 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:31.012622 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 12:20:31.012979 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":471,"threads_started":1}
I20260811 12:20:31.021786 4583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97783fd1ef2242539b4bfadb70e017f0" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44893 } }
I20260811 12:20:31.022280 4583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:31.022387 4583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97783fd1ef2242539b4bfadb70e017f0, State: Initialized, Role: FOLLOWER
I20260811 12:20:31.023078 4583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97783fd1ef2242539b4bfadb70e017f0" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44893 } }
I20260811 12:20:31.023358 4583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:31.023516 4583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:31.023694 4583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:31.026201 4583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97783fd1ef2242539b4bfadb70e017f0" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44893 } }
I20260811 12:20:31.026800 4583 leader_election.cc:302] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97783fd1ef2242539b4bfadb70e017f0; no voters:
I20260811 12:20:31.027444 4583 leader_election.cc:288] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:31.027594 4589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:31.028268 4589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [term 1 LEADER]: Becoming Leader. State: Replica: 97783fd1ef2242539b4bfadb70e017f0, State: Running, Role: LEADER
I20260811 12:20:31.028870 4589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97783fd1ef2242539b4bfadb70e017f0" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44893 } }
I20260811 12:20:31.029616 4583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:31.034782 4591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97783fd1ef2242539b4bfadb70e017f0. Latest consensus state: current_term: 1 leader_uuid: "97783fd1ef2242539b4bfadb70e017f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97783fd1ef2242539b4bfadb70e017f0" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44893 } } }
I20260811 12:20:31.035265 4591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:31.034750 4590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97783fd1ef2242539b4bfadb70e017f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97783fd1ef2242539b4bfadb70e017f0" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 44893 } } }
I20260811 12:20:31.035413 4590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:31.036936 4605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:31.042168 4605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:31.052402 4605 catalog_manager.cc:1347] Generated new cluster ID: 2ad115f246844cad854ea33a33ee3544
I20260811 12:20:31.052523 4605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:31.065799 4605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:31.067368 4605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:31.075845 4605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0: Generated new TSK 0
I20260811 12:20:31.077101 4605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:20:31.164702 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44893
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:40115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:31.523195 4622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:31.523488 4622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:31.523592 4622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:31.531095 4622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:31.531243 4622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:31.531371 4622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:20:31.542114 4622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:40115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:31.543638 4622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:31.545091 4622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:31.558930 4630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:31.558882 4629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:31.558853 4632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:31.559737 4622 server_base.cc:1034] running on GCE node
I20260811 12:20:31.560312 4622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:31.561348 4622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:31.562546 4622 hybrid_clock.cc:648] HybridClock initialized: now 1786450831562488 us; error 54 us; skew 500 ppm
I20260811 12:20:31.562882 4622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:31 dist-test-slave-rmhg krb5kdc[4453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450831, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:31.570272 4622 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:20:31.571959 4622 webserver.cc:460] Webserver started at http://127.7.39.1:45039/ using document root <none> and password file <none>
I20260811 12:20:31.572710 4622 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:31.572836 4622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:31.573189 4622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:31.576421 4622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "83ef62dae6794aa08185dbe1c639ba8c"
format_stamp: "Formatted at 2026-08-11 12:20:31 on dist-test-slave-rmhg"
server_key: "ab483851464096ef1a2a82c5b44a9141"
server_key_iv: "558f034ef209794451ab19b5f41c199b"
server_key_version: "encryptionkey@0"
I20260811 12:20:31.577365 4622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "83ef62dae6794aa08185dbe1c639ba8c"
format_stamp: "Formatted at 2026-08-11 12:20:31 on dist-test-slave-rmhg"
server_key: "ab483851464096ef1a2a82c5b44a9141"
server_key_iv: "558f034ef209794451ab19b5f41c199b"
server_key_version: "encryptionkey@0"
I20260811 12:20:31.583220 4622 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:20:31.587637 4643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:31.589219 4622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:20:31.589401 4622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "83ef62dae6794aa08185dbe1c639ba8c"
format_stamp: "Formatted at 2026-08-11 12:20:31 on dist-test-slave-rmhg"
server_key: "ab483851464096ef1a2a82c5b44a9141"
server_key_iv: "558f034ef209794451ab19b5f41c199b"
server_key_version: "encryptionkey@0"
I20260811 12:20:31.589686 4622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:31.616089 4622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:31.620605 4622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:31.620921 4622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:31.622073 4622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:31.623602 4622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:31.623689 4622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:31.623829 4622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:31.623885 4622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:31.659642 4622 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:39689
I20260811 12:20:31.659722 4832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:39689 every 8 connection(s)
I20260811 12:20:31.661046 4622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:20:31.669106 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 4622
I20260811 12:20:31.669318 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:20:31.669777 7324 external_mini_cluster.cc:1428] Setting key 8162127b6c6abcc53000a8ef9e60bb6b
Aug 11 12:20:31 dist-test-slave-rmhg krb5kdc[4453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450831, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:31.682819 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:31.664036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:49017 (local address 127.7.39.62:44893)
0811 12:20:31.664550 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:31.664555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:31.665759 (+ 1204us) server_negotiation.cc:407] Connection header received
0811 12:20:31.667788 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:31.667794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:31.667885 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:31.667985 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:31.669884 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:31.670757 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:31.672439 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:31.672857 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:31.672993 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:31.676888 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:31.676895 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:31.676898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:31.676939 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:31.679449 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:31.680275 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:31.680280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:31.680283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:31.680420 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:31.680891 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:31.680895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:31.680897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:31.681246 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:31.681392 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:31.682200 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 12:20:31.682444 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":247,"threads_started":1}
I20260811 12:20:31.683880 4842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:31.664705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44893 (local address 127.7.39.1:49017)
0811 12:20:31.665556 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:31.665618 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:31.667470 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:31.668322 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:31.668333 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:31.668947 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:31.669694 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:31.669715 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:31.670943 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:31.670947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:31.672283 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:31.672296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:31.673104 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:31.673111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:31.673408 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:31.674185 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:31.674212 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:31.676620 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:31.679624 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:31.679635 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:31.679655 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:31.680135 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:31.680538 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:31.680543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:31.680547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:31.680767 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:31.681386 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:31.681395 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:31.681775 (+ 380us) client_negotiation.cc:769] Sending connection context
0811 12:20:31.682047 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 12:20:31.682639 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":333,"threads_started":1}
I20260811 12:20:31.685611 4835 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44893
I20260811 12:20:31.685938 4835 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:31.686679 4835 heartbeater.cc:511] Master 127.7.39.62:44893 requested a full tablet report, sending...
I20260811 12:20:31.688876 4530 ts_manager.cc:194] Registered new tserver with Master: 83ef62dae6794aa08185dbe1c639ba8c (127.7.39.1:39689)
I20260811 12:20:31.690444 4530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:49017
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:20:31.740239 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44893
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:40115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:32.092634 4850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:32.092957 4850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:32.093111 4850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:32.100787 4850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:32.100939 4850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:32.101099 4850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:20:32.111333 4850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:40115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:32.112769 4850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:32.114253 4850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:32.128103 4856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:32.128089 4857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:32.128551 4862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:32.129118 4850 server_base.cc:1034] running on GCE node
I20260811 12:20:32.129839 4850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:32.130975 4850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:32.132194 4850 hybrid_clock.cc:648] HybridClock initialized: now 1786450832132130 us; error 56 us; skew 500 ppm
I20260811 12:20:32.132520 4850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:32 dist-test-slave-rmhg krb5kdc[4453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:32.136000 4850 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:20:32.137652 4850 webserver.cc:460] Webserver started at http://127.7.39.2:34059/ using document root <none> and password file <none>
I20260811 12:20:32.138337 4850 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:32.138442 4850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:32.138737 4850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:32.142112 4850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "8c797b33157a4dae8709ff589eaf84e5"
format_stamp: "Formatted at 2026-08-11 12:20:32 on dist-test-slave-rmhg"
server_key: "3b9ee7ebcb1527b1eb28e64e9f342bae"
server_key_iv: "38884ad60eed037f5a8b1bbd98b9357a"
server_key_version: "encryptionkey@0"
I20260811 12:20:32.143066 4850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "8c797b33157a4dae8709ff589eaf84e5"
format_stamp: "Formatted at 2026-08-11 12:20:32 on dist-test-slave-rmhg"
server_key: "3b9ee7ebcb1527b1eb28e64e9f342bae"
server_key_iv: "38884ad60eed037f5a8b1bbd98b9357a"
server_key_version: "encryptionkey@0"
I20260811 12:20:32.148924 4850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:20:32.153079 4872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:32.154635 4850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:20:32.154834 4850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "8c797b33157a4dae8709ff589eaf84e5"
format_stamp: "Formatted at 2026-08-11 12:20:32 on dist-test-slave-rmhg"
server_key: "3b9ee7ebcb1527b1eb28e64e9f342bae"
server_key_iv: "38884ad60eed037f5a8b1bbd98b9357a"
server_key_version: "encryptionkey@0"
I20260811 12:20:32.155045 4850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:32.182699 4850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:32.187995 4850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:32.188326 4850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:32.189451 4850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:32.190930 4850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:32.191022 4850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:32.191146 4850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:32.191200 4850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:32.224354 4850 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:44161
I20260811 12:20:32.224490 5063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:44161 every 8 connection(s)
I20260811 12:20:32.225852 4850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:20:32.234943 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 4850
I20260811 12:20:32.235164 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:20:32.235762 7324 external_mini_cluster.cc:1428] Setting key 11b4cdc1e13f0d9bc102cc64b51e0184
Aug 11 12:20:32 dist-test-slave-rmhg krb5kdc[4453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:32.248929 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.228491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:58319 (local address 127.7.39.62:44893)
0811 12:20:32.228969 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:32.228975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:32.230214 (+ 1239us) server_negotiation.cc:407] Connection header received
0811 12:20:32.232254 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:32.232259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:32.232350 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:32.232462 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:32.234287 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.235233 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.237334 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.237838 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.237997 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.242361 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:32.242369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:32.242372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:32.242412 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:32.245434 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:32.246344 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:32.246348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:32.246351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:32.246498 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:32.247026 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:32.247031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:32.247033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:32.247342 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:32.247470 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:32.248437 (+ 967us) server_negotiation.cc:299] Negotiation successful
0811 12:20:32.248650 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":213,"threads_started":1}
I20260811 12:20:32.250279 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.229089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44893 (local address 127.7.39.2:58319)
0811 12:20:32.230017 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:32.230072 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:32.231910 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:32.232723 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:32.232731 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:32.233388 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:32.234068 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.234095 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.235403 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.235409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.237179 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.237197 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.238087 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.238092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.238388 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.239200 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:32.239229 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:32.242073 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:32.245594 (+ 3521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:32.245604 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:32.245625 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:32.246220 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:32.246609 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:32.246613 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:32.246616 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:32.246867 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:32.247475 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:32.247483 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:32.247949 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 12:20:32.248267 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 12:20:32.248824 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":362,"threads_started":1}
I20260811 12:20:32.251849 5066 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44893
I20260811 12:20:32.252208 5066 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:32.253204 5066 heartbeater.cc:511] Master 127.7.39.62:44893 requested a full tablet report, sending...
I20260811 12:20:32.255002 4530 ts_manager.cc:194] Registered new tserver with Master: 8c797b33157a4dae8709ff589eaf84e5 (127.7.39.2:44161)
I20260811 12:20:32.255897 4530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:58319
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:20:32.312780 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44893
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:40115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:32.670215 5076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:32.670545 5076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:32.670701 5076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:32.678207 5076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:32.678325 5076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:32.678484 5076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:20:32.688481 5076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:40115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:44893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:32.689888 5076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:32.691403 5076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 12:20:32.693630 4835 heartbeater.cc:503] Master 127.7.39.62:44893 was elected leader, sending a full tablet report...
W20260811 12:20:32.705242 5087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:32.705278 5082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:32.705319 5081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:32.710760 5076 server_base.cc:1034] running on GCE node
I20260811 12:20:32.711462 5076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:32.712504 5076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:32.713738 5076 hybrid_clock.cc:648] HybridClock initialized: now 1786450832713696 us; error 38 us; skew 500 ppm
I20260811 12:20:32.714092 5076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:32 dist-test-slave-rmhg krb5kdc[4453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:32.717543 5076 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:20:32.719187 5076 webserver.cc:460] Webserver started at http://127.7.39.3:39663/ using document root <none> and password file <none>
I20260811 12:20:32.719998 5076 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:32.720106 5076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:32.720379 5076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:32.723505 5076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "6d2b697579b640468daca57c0a3b09d5"
format_stamp: "Formatted at 2026-08-11 12:20:32 on dist-test-slave-rmhg"
server_key: "534a1e2f71cbb2c1f66d4d1adfab9e95"
server_key_iv: "25dab51a1e987cb26a7b22598d9a1710"
server_key_version: "encryptionkey@0"
I20260811 12:20:32.724324 5076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "6d2b697579b640468daca57c0a3b09d5"
format_stamp: "Formatted at 2026-08-11 12:20:32 on dist-test-slave-rmhg"
server_key: "534a1e2f71cbb2c1f66d4d1adfab9e95"
server_key_iv: "25dab51a1e987cb26a7b22598d9a1710"
server_key_version: "encryptionkey@0"
I20260811 12:20:32.730206 5076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:20:32.734011 5097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:32.735598 5076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:32.735756 5076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "6d2b697579b640468daca57c0a3b09d5"
format_stamp: "Formatted at 2026-08-11 12:20:32 on dist-test-slave-rmhg"
server_key: "534a1e2f71cbb2c1f66d4d1adfab9e95"
server_key_iv: "25dab51a1e987cb26a7b22598d9a1710"
server_key_version: "encryptionkey@0"
I20260811 12:20:32.735970 5076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:32.753728 5076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:32.758061 5076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:32.758342 5076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:32.759349 5076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:32.760711 5076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:32.760787 5076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:32.760902 5076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:32.760951 5076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:32.793062 5076 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:37019
I20260811 12:20:32.793144 5268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:37019 every 8 connection(s)
I20260811 12:20:32.794490 5076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:20:32.796723 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 5076
I20260811 12:20:32.796919 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:20:32.797577 7324 external_mini_cluster.cc:1428] Setting key 796034055be198ebdc476730f581b4bf
Aug 11 12:20:32 dist-test-slave-rmhg krb5kdc[4453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:32.815357 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.797230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:46763 (local address 127.7.39.62:44893)
0811 12:20:32.797647 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:32.797653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:32.799039 (+ 1386us) server_negotiation.cc:407] Connection header received
0811 12:20:32.800914 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:32.800918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:32.800988 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:32.801084 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:32.802695 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.803465 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.805077 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.805540 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.805639 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.809101 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:32.809106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:32.809110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:32.809197 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:32.812275 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:32.813224 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:32.813231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:32.813235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:32.813373 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:32.813759 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:32.813762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:32.813764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:32.814023 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:32.814136 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:32.814892 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 12:20:32.815085 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":205,"threads_started":1}
I20260811 12:20:32.816385 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.797973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:44893 (local address 127.7.39.3:46763)
0811 12:20:32.798855 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:32.798918 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:32.800654 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:32.801338 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:32.801347 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:32.801940 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:32.802538 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.802556 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.803623 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.803627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.804931 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.804946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.805741 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.805746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.805945 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.806564 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:32.806592 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:32.808851 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:32.812425 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:32.812434 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:32.812489 (+ 55us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:32.813071 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:32.813485 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:32.813489 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:32.813491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:32.813680 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:32.814184 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:32.814191 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:32.814535 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 12:20:32.814792 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 12:20:32.815203 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":360,"threads_started":1}
I20260811 12:20:32.817831 5269 heartbeater.cc:347] Connected to a master server at 127.7.39.62:44893
I20260811 12:20:32.818156 5269 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:32.818948 5269 heartbeater.cc:511] Master 127.7.39.62:44893 requested a full tablet report, sending...
I20260811 12:20:32.820452 4530 ts_manager.cc:194] Registered new tserver with Master: 6d2b697579b640468daca57c0a3b09d5 (127.7.39.3:37019)
I20260811 12:20:32.821367 4530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:46763
I20260811 12:20:32.825650 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:20:32.843503 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.829993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.7.39.62:44893)
0811 12:20:32.830162 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:32.830167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:32.830654 (+ 487us) server_negotiation.cc:407] Connection header received
0811 12:20:32.830813 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:32.830816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:32.830900 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:32.831020 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:32.832486 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.833269 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.834706 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.835113 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.835234 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.838086 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:32.838091 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:32.838094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:32.838128 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:32.840754 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:32.841700 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:32.841704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:32.841706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:32.841833 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:32.842216 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:32.842220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:32.842222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:32.842532 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:32.842831 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:32.843111 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 12:20:32.843273 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:20:32.847637 4530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:20:32.854660 4530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:20:32.891515 5286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.867821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.7.39.3:37019)
0811 12:20:32.883475 (+ 15654us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:32.883484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:32.883515 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:20:32.883606 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:32.883613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:32.883847 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:32.884878 (+ 1031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:32.885207 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.886664 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.888866 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.890322 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.890442 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.890727 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:32.890958 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 12:20:32.891131 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15456,"thread_start_us":15291,"threads_started":1}
I20260811 12:20:32.891610 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.867649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:37019 (local address 127.0.0.1:45682)
0811 12:20:32.880003 (+ 12354us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:32.880085 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:32.880342 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:32.884096 (+ 3754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:32.884101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:32.884131 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:32.884671 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.884684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.886804 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.886810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.888651 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.888665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.890501 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.890508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.890711 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.890768 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 12:20:32.890897 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 12:20:32.891132 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11996,"mutex_wait_us":178,"thread_start_us":15048,"threads_started":1}
I20260811 12:20:32.892108 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.867934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39689 (local address 127.0.0.1:47664)
0811 12:20:32.882051 (+ 14117us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:32.882073 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:32.882234 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:32.883465 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:32.883472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:32.883505 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:32.884331 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.884346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.886445 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.886450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.888896 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.888917 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.891138 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.891143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.891344 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.891445 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 12:20:32.891535 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 12:20:32.891694 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13972,"mutex_wait_us":213,"thread_start_us":13808,"threads_started":1}
I20260811 12:20:32.892149 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.869877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.7.39.2:44161)
0811 12:20:32.882511 (+ 12634us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:32.882522 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:32.883311 (+ 789us) server_negotiation.cc:407] Connection header received
0811 12:20:32.883423 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:32.883429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:32.883628 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:32.883808 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:32.884642 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.886060 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.889112 (+ 3052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.890624 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.890752 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.890942 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:32.891661 (+ 719us) server_negotiation.cc:299] Negotiation successful
0811 12:20:32.891840 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12423,"thread_start_us":12277,"threads_started":1}
I20260811 12:20:32.892598 5285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.869640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:44161 (local address 127.0.0.1:38550)
0811 12:20:32.883005 (+ 13365us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:32.883028 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:32.883178 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:32.883835 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:32.883839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:32.883872 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:32.884364 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.884387 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.886199 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.886204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.888920 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:32.888930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.891111 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.891121 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:32.891331 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.891451 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 12:20:32.891534 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:20:32.892221 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13230,"mutex_wait_us":678,"thread_start_us":9812,"threads_started":1}
I20260811 12:20:32.893414 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:32.868602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.7.39.1:39689)
0811 12:20:32.877960 (+ 9358us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:32.877971 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:32.882388 (+ 4417us) server_negotiation.cc:407] Connection header received
0811 12:20:32.882530 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:32.882540 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:32.882904 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:32.883151 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:32.884601 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.886304 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.889113 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:32.890622 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:32.890752 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:32.890986 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:32.892840 (+ 1854us) server_negotiation.cc:299] Negotiation successful
0811 12:20:32.893025 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9129,"thread_start_us":8936,"threads_started":1}
I20260811 12:20:32.896385 5176 tablet_service.cc:1467] Processing CreateTablet for tablet 523fadf7136d47328e6137cc0267f7fc (DEFAULT_TABLE table=test-table [id=19ac91a917a54f21933c1857e8d47ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:32.897056 4961 tablet_service.cc:1467] Processing CreateTablet for tablet 523fadf7136d47328e6137cc0267f7fc (DEFAULT_TABLE table=test-table [id=19ac91a917a54f21933c1857e8d47ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:32.897058 4723 tablet_service.cc:1467] Processing CreateTablet for tablet 523fadf7136d47328e6137cc0267f7fc (DEFAULT_TABLE table=test-table [id=19ac91a917a54f21933c1857e8d47ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:32.897852 5176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 523fadf7136d47328e6137cc0267f7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:32.898541 4961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 523fadf7136d47328e6137cc0267f7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:32.898559 4723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 523fadf7136d47328e6137cc0267f7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:32.911222 5293 tablet_bootstrap.cc:492] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Bootstrap starting.
I20260811 12:20:32.913779 5295 tablet_bootstrap.cc:492] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Bootstrap starting.
I20260811 12:20:32.915393 5293 tablet_bootstrap.cc:654] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:32.916873 5293 log.cc:824] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:32.917732 5295 tablet_bootstrap.cc:654] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:32.919266 5295 log.cc:824] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:32.920495 5297 tablet_bootstrap.cc:492] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Bootstrap starting.
I20260811 12:20:32.923024 5295 tablet_bootstrap.cc:492] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: No bootstrap required, opened a new log
I20260811 12:20:32.923374 5295 ts_tablet_manager.cc:1397] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 12:20:32.924502 5297 tablet_bootstrap.cc:654] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:32.924976 5293 tablet_bootstrap.cc:492] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: No bootstrap required, opened a new log
I20260811 12:20:32.925294 5293 ts_tablet_manager.cc:1397] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260811 12:20:32.925908 5297 log.cc:824] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:32.929984 5297 tablet_bootstrap.cc:492] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: No bootstrap required, opened a new log
I20260811 12:20:32.930302 5297 ts_tablet_manager.cc:1397] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:20:32.933341 5293 raft_consensus.cc:348] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:32.933754 5293 raft_consensus.cc:374] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:32.933856 5293 raft_consensus.cc:729] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d2b697579b640468daca57c0a3b09d5, State: Initialized, Role: FOLLOWER
I20260811 12:20:32.933849 5295 raft_consensus.cc:348] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:32.934351 5295 raft_consensus.cc:374] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:32.934471 5295 raft_consensus.cc:729] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef62dae6794aa08185dbe1c639ba8c, State: Initialized, Role: FOLLOWER
I20260811 12:20:32.934508 5293 consensus_queue.cc:260] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:32.935143 5295 consensus_queue.cc:260] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:32.935770 5269 heartbeater.cc:503] Master 127.7.39.62:44893 was elected leader, sending a full tablet report...
I20260811 12:20:32.936998 5295 ts_tablet_manager.cc:1428] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260811 12:20:32.938258 5293 ts_tablet_manager.cc:1428] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Time spent starting tablet: real 0.013s user 0.006s sys 0.004s
I20260811 12:20:32.940825 5297 raft_consensus.cc:348] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:32.941300 5297 raft_consensus.cc:374] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:32.941413 5297 raft_consensus.cc:729] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c797b33157a4dae8709ff589eaf84e5, State: Initialized, Role: FOLLOWER
I20260811 12:20:32.942274 5297 consensus_queue.cc:260] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:32.945938 5066 heartbeater.cc:503] Master 127.7.39.62:44893 was elected leader, sending a full tablet report...
I20260811 12:20:32.946835 5297 ts_tablet_manager.cc:1428] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
W20260811 12:20:32.977914 5067 tablet.cc:2369] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:20:33.046202 5270 tablet.cc:2369] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:33.141955 5306 raft_consensus.cc:482] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:33.142333 5306 raft_consensus.cc:504] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:33.144249 5306 leader_election.cc:288] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d2b697579b640468daca57c0a3b09d5 (127.7.39.3:37019), 83ef62dae6794aa08185dbe1c639ba8c (127.7.39.1:39689)
I20260811 12:20:33.154194 5286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:33.144771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:36459 (local address 127.7.39.3:37019)
0811 12:20:33.145000 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:33.145007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:33.145981 (+ 974us) server_negotiation.cc:407] Connection header received
0811 12:20:33.146164 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:33.146169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:33.146254 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:33.146360 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:33.147185 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.148407 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.151612 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.152894 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.153035 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:33.153186 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:33.153763 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 12:20:33.153913 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:20:33.154194 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:33.144786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:38449 (local address 127.7.39.1:39689)
0811 12:20:33.145000 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:33.145006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:33.145962 (+ 956us) server_negotiation.cc:407] Connection header received
0811 12:20:33.146164 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:33.146169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:33.146254 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:33.146360 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:33.147263 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.148406 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.151630 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.152881 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.153036 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:33.153184 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:33.153763 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 12:20:33.153919 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:20:33.154448 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:33.144845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:37019 (local address 127.7.39.2:36459)
0811 12:20:33.145809 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:33.145831 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:33.146018 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:33.146426 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:33.146432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:33.146483 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:33.146974 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:33.146993 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.148595 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.148599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:33.151410 (+ 2811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:33.151430 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.153231 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.153238 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:33.153482 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:33.153606 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 12:20:33.153717 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:20:33.153953 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":207,"spinlock_wait_cycles":73344,"thread_start_us":514,"threads_started":1}
I20260811 12:20:33.154978 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:33.144845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:39689 (local address 127.7.39.2:38449)
0811 12:20:33.145848 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:33.145863 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:33.146033 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:33.146426 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:33.146432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:33.146485 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:33.147085 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:33.147097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.148593 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.148599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:33.151410 (+ 2811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:33.151430 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.153230 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.153238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:33.153482 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:33.153627 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 12:20:33.153716 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:20:33.153953 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":456,"spinlock_wait_cycles":38528,"thread_start_us":474,"threads_started":1}
I20260811 12:20:33.155550 5204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523fadf7136d47328e6137cc0267f7fc" candidate_uuid: "8c797b33157a4dae8709ff589eaf84e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d2b697579b640468daca57c0a3b09d5" is_pre_election: true
I20260811 12:20:33.156203 5204 raft_consensus.cc:2454] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c797b33157a4dae8709ff589eaf84e5 in term 0.
I20260811 12:20:33.156061 4758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523fadf7136d47328e6137cc0267f7fc" candidate_uuid: "8c797b33157a4dae8709ff589eaf84e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef62dae6794aa08185dbe1c639ba8c" is_pre_election: true
I20260811 12:20:33.156674 4758 raft_consensus.cc:2454] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c797b33157a4dae8709ff589eaf84e5 in term 0.
I20260811 12:20:33.157049 4873 leader_election.cc:302] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d2b697579b640468daca57c0a3b09d5, 8c797b33157a4dae8709ff589eaf84e5; no voters:
I20260811 12:20:33.157521 5306 raft_consensus.cc:2781] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:20:33.157670 5306 raft_consensus.cc:482] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:20:33.157771 5306 raft_consensus.cc:3037] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:33.160177 5306 raft_consensus.cc:504] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:33.160791 5306 leader_election.cc:288] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [CANDIDATE]: Term 1 election: Requested vote from peers 6d2b697579b640468daca57c0a3b09d5 (127.7.39.3:37019), 83ef62dae6794aa08185dbe1c639ba8c (127.7.39.1:39689)
I20260811 12:20:33.161386 5204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523fadf7136d47328e6137cc0267f7fc" candidate_uuid: "8c797b33157a4dae8709ff589eaf84e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d2b697579b640468daca57c0a3b09d5"
I20260811 12:20:33.161386 4758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523fadf7136d47328e6137cc0267f7fc" candidate_uuid: "8c797b33157a4dae8709ff589eaf84e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ef62dae6794aa08185dbe1c639ba8c"
I20260811 12:20:33.161592 5204 raft_consensus.cc:3037] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:33.161592 4758 raft_consensus.cc:3037] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:33.163851 4758 raft_consensus.cc:2454] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c797b33157a4dae8709ff589eaf84e5 in term 1.
I20260811 12:20:33.163859 5204 raft_consensus.cc:2454] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c797b33157a4dae8709ff589eaf84e5 in term 1.
W20260811 12:20:33.163877 4836 tablet.cc:2369] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:33.164372 4879 leader_election.cc:302] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef62dae6794aa08185dbe1c639ba8c, 8c797b33157a4dae8709ff589eaf84e5; no voters:
I20260811 12:20:33.164726 5306 raft_consensus.cc:2781] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:33.165222 5306 raft_consensus.cc:686] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 1 LEADER]: Becoming Leader. State: Replica: 8c797b33157a4dae8709ff589eaf84e5, State: Running, Role: LEADER
I20260811 12:20:33.165709 5306 consensus_queue.cc:237] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } }
I20260811 12:20:33.170327 4530 catalog_manager.cc:5495] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c797b33157a4dae8709ff589eaf84e5 (127.7.39.2). New cstate: current_term: 1 leader_uuid: "8c797b33157a4dae8709ff589eaf84e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c797b33157a4dae8709ff589eaf84e5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 44161 } health_report { overall_health: HEALTHY } } }
I20260811 12:20:33.187167 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:33.181769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.7.39.2:44161)
0811 12:20:33.181958 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:33.181963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:33.181982 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:20:33.182084 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:33.182088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:33.182176 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:33.182304 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:20:33.182663 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.183401 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.184846 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.185323 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.185420 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:33.185940 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:20:33.186470 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:20:33.186607 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:33.186800 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:20:33.186847 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:20:33.195648 5204 raft_consensus.cc:1261] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 1 FOLLOWER]: Refusing update from remote peer 8c797b33157a4dae8709ff589eaf84e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:33.195678 4758 raft_consensus.cc:1261] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 1 FOLLOWER]: Refusing update from remote peer 8c797b33157a4dae8709ff589eaf84e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:33.196676 5313 consensus_queue.cc:1035] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d2b697579b640468daca57c0a3b09d5" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 37019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:33.197018 5306 consensus_queue.cc:1035] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ef62dae6794aa08185dbe1c639ba8c" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 39689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:33.209576 5323 mvcc.cc:205] Tried to move back new op lower bound from 7317302612759207936 to 7317302612650594304. Current Snapshot: MvccSnapshot[applied={T|T < 7317302612759207936}]
I20260811 12:20:33.224956 4530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:33.225277 4530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:33.229265 4530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97783fd1ef2242539b4bfadb70e017f0: Sending DeleteTablet for 3 replicas of tablet 523fadf7136d47328e6137cc0267f7fc
I20260811 12:20:33.230564 4723 tablet_service.cc:1514] Processing DeleteTablet for tablet 523fadf7136d47328e6137cc0267f7fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:33 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:47664
I20260811 12:20:33.230563 5176 tablet_service.cc:1514] Processing DeleteTablet for tablet 523fadf7136d47328e6137cc0267f7fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:33 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:45682
I20260811 12:20:33.233682 4961 tablet_service.cc:1514] Processing DeleteTablet for tablet 523fadf7136d47328e6137cc0267f7fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:33 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:38550
I20260811 12:20:33.234730 5335 tablet_replica.cc:330] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: stopping tablet replica
I20260811 12:20:33.235272 5335 raft_consensus.cc:2229] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:33.235843 5335 raft_consensus.cc:2258] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:33.236387 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 4622
I20260811 12:20:33.238782 5335 ts_tablet_manager.cc:1905] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:33.247864 5334 tablet_replica.cc:330] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: stopping tablet replica
I20260811 12:20:33.248353 5334 raft_consensus.cc:2229] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:33.248647 5337 tablet_replica.cc:330] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: stopping tablet replica
I20260811 12:20:33.248943 5334 raft_consensus.cc:2258] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:33.249234 5337 raft_consensus.cc:2229] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:20:33.249982 5337 raft_consensus.cc:2258] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:33.251464 5334 ts_tablet_manager.cc:1905] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:33.252180 5337 ts_tablet_manager.cc:1905] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:33.256801 5335 ts_tablet_manager.cc:1918] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:33.258235 5334 ts_tablet_manager.cc:1918] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:33.258394 5334 log.cc:1196] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/523fadf7136d47328e6137cc0267f7fc
I20260811 12:20:33.258978 5334 ts_tablet_manager.cc:1939] T 523fadf7136d47328e6137cc0267f7fc P 6d2b697579b640468daca57c0a3b09d5: Deleting consensus metadata
I20260811 12:20:33.260890 4506 catalog_manager.cc:4841] TS 6d2b697579b640468daca57c0a3b09d5 (127.7.39.3:37019): tablet 523fadf7136d47328e6137cc0267f7fc (table test-table [id=19ac91a917a54f21933c1857e8d47ec7]) successfully deleted
I20260811 12:20:33.261250 5335 log.cc:1196] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/523fadf7136d47328e6137cc0267f7fc
I20260811 12:20:33.261814 5335 ts_tablet_manager.cc:1939] T 523fadf7136d47328e6137cc0267f7fc P 83ef62dae6794aa08185dbe1c639ba8c: Deleting consensus metadata
I20260811 12:20:33.265211 5337 ts_tablet_manager.cc:1918] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 11 12:20:33 dist-test-slave-rmhg krb5kdc[4453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:20:33.265404 5337 log.cc:1196] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/523fadf7136d47328e6137cc0267f7fc
I20260811 12:20:33.266006 5337 ts_tablet_manager.cc:1939] T 523fadf7136d47328e6137cc0267f7fc P 8c797b33157a4dae8709ff589eaf84e5: Deleting consensus metadata
I20260811 12:20:33.267714 4511 catalog_manager.cc:4841] TS 83ef62dae6794aa08185dbe1c639ba8c (127.7.39.1:39689): tablet 523fadf7136d47328e6137cc0267f7fc (table test-table [id=19ac91a917a54f21933c1857e8d47ec7]) successfully deleted
I20260811 12:20:33.268072 4510 catalog_manager.cc:4841] TS 8c797b33157a4dae8709ff589eaf84e5 (127.7.39.2:44161): tablet 523fadf7136d47328e6137cc0267f7fc (table test-table [id=19ac91a917a54f21933c1857e8d47ec7]) successfully deleted
I20260811 12:20:33.273301 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:33.237620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.7.39.1:39689)
0811 12:20:33.237803 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:33.237809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:33.245685 (+ 7876us) server_negotiation.cc:407] Connection header received
0811 12:20:33.246008 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:33.246015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:33.246119 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:33.246198 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:33.248079 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.249218 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.259989 (+ 10771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.260650 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.260759 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:33.266752 (+ 5993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:33.266766 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:33.266784 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:33.266839 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:33.270328 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:33.271035 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:33.271043 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:33.271046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:33.271184 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:33.271641 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:33.271645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:33.271647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:33.272035 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:33.272320 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:33.272655 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 12:20:33.272872 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:20:33.273939 4825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:33.426421 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 4622
I20260811 12:20:33.451469 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 4850
Aug 11 12:20:33 dist-test-slave-rmhg krb5kdc[4453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:20:33.470907 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:33.452294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.7.39.2:44161)
0811 12:20:33.452480 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:33.452486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:33.452721 (+ 235us) server_negotiation.cc:407] Connection header received
0811 12:20:33.452896 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:33.452900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:33.452996 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:33.453105 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:33.454899 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.455721 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.457898 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.458502 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.458615 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:33.464001 (+ 5386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:33.464019 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:33.464039 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:33.464088 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:33.467686 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:33.468555 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:33.468565 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:33.468567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:33.468759 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:33.469221 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:33.469225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:33.469227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:33.469612 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:33.469988 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:33.470309 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 12:20:33.470533 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:20:33.471608 5053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:33.591230 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 4850
I20260811 12:20:33.617435 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 5076
Aug 11 12:20:33 dist-test-slave-rmhg krb5kdc[4453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:20:33.633971 5286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:33.618287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.7.39.3:37019)
0811 12:20:33.618482 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:33.618488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:33.618508 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:20:33.618716 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:33.618723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:33.618794 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:33.618881 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:33.620233 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.620954 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.622384 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:33.622804 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:33.622923 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:33.627719 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:33.627739 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:33.627759 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:33.627811 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:33.630772 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:33.631614 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:33.631622 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:33.631625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:33.631796 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:33.632207 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:33.632212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:33.632214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:33.632591 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:33.632875 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:33.633298 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 12:20:33.633528 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:20:33.634713 5264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:33.723165 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 5076
I20260811 12:20:33.747223 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 4479
I20260811 12:20:33.748375 4575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:33.860467 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 4479
2026-08-11T12:20:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4958 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T12:20:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:33Z Disabled control of system clock
I20260811 12:20:33.962134 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35547
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:35547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:20:34.320547 5349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:34.320889 5349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:34.320978 5349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:34.328613 5349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:34.328748 5349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:34.328788 5349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:34.328840 5349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:34.328892 5349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:34.339298 5349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:35547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:34.340747 5349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:34.342780 5349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:34.355897 5357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:34.356020 5361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:34.355825 5358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:34.357041 5349 server_base.cc:1034] running on GCE node
I20260811 12:20:34.357962 5349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:34.359490 5349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:34.360726 5349 hybrid_clock.cc:648] HybridClock initialized: now 1786450834360668 us; error 51 us; skew 500 ppm
I20260811 12:20:34.361109 5349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:34.363538 5349 webserver.cc:460] Webserver started at http://127.7.39.62:40371/ using document root <none> and password file <none>
I20260811 12:20:34.364272 5349 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:34.364378 5349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:34.364732 5349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:34.367909 5349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "7e0cf459aa654a33a39913ecd6772408"
format_stamp: "Formatted at 2026-08-11 12:20:34 on dist-test-slave-rmhg"
server_key: "049593c263f507c40086be7cc3404dab"
server_key_iv: "14b89563a78e0244a1cdef1e8d2e4c44"
server_key_version: "encryptionkey@0"
I20260811 12:20:34.368755 5349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "7e0cf459aa654a33a39913ecd6772408"
format_stamp: "Formatted at 2026-08-11 12:20:34 on dist-test-slave-rmhg"
server_key: "049593c263f507c40086be7cc3404dab"
server_key_iv: "14b89563a78e0244a1cdef1e8d2e4c44"
server_key_version: "encryptionkey@0"
I20260811 12:20:34.374167 5349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:20:34.377893 5370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:34.379858 5349 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:20:34.380046 5349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "7e0cf459aa654a33a39913ecd6772408"
format_stamp: "Formatted at 2026-08-11 12:20:34 on dist-test-slave-rmhg"
server_key: "049593c263f507c40086be7cc3404dab"
server_key_iv: "14b89563a78e0244a1cdef1e8d2e4c44"
server_key_version: "encryptionkey@0"
I20260811 12:20:34.380319 5349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:34.413367 5349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:34.414318 5349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:34.414629 5349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:34.433712 5349 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:35547
I20260811 12:20:34.433755 5446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:35547 every 8 connection(s)
I20260811 12:20:34.435205 5349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:34.435725 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 5349
I20260811 12:20:34.435897 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:34.436532 7324 external_mini_cluster.cc:1428] Setting key 2ebfb9e849df2dee2aac9456e96a6781
I20260811 12:20:34.440171 5447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:34.453706 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:34.438652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37046 (local address 127.7.39.62:35547)
0811 12:20:34.439798 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:34.439813 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:34.439868 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:20:34.442217 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:34.442248 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:34.442875 (+ 627us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:34.443360 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:34.443972 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:34.446172 (+ 2200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:34.447794 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:34.448892 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:34.449084 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:34.450435 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:34.450446 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:34.450468 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:34.451306 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:34.451418 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:34.451448 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:34.451592 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:34.451810 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:20:34.452230 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":359,"threads_started":1}
I20260811 12:20:34.455017 5447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408: Bootstrap starting.
I20260811 12:20:34.458829 5447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:34.459900 5447 log.cc:824] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:34.463287 5447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408: No bootstrap required, opened a new log
I20260811 12:20:34.471246 5447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0cf459aa654a33a39913ecd6772408" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35547 } }
I20260811 12:20:34.471662 5447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:34.471745 5447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0cf459aa654a33a39913ecd6772408, State: Initialized, Role: FOLLOWER
I20260811 12:20:34.472302 5447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0cf459aa654a33a39913ecd6772408" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35547 } }
I20260811 12:20:34.472514 5447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:34.472631 5447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:34.472787 5447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:34.475548 5447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0cf459aa654a33a39913ecd6772408" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35547 } }
I20260811 12:20:34.476071 5447 leader_election.cc:302] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e0cf459aa654a33a39913ecd6772408; no voters:
I20260811 12:20:34.476639 5447 leader_election.cc:288] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:34.476787 5456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:34.477516 5456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [term 1 LEADER]: Becoming Leader. State: Replica: 7e0cf459aa654a33a39913ecd6772408, State: Running, Role: LEADER
I20260811 12:20:34.478031 5456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0cf459aa654a33a39913ecd6772408" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35547 } }
I20260811 12:20:34.478749 5447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:34.480880 5457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e0cf459aa654a33a39913ecd6772408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0cf459aa654a33a39913ecd6772408" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35547 } } }
I20260811 12:20:34.481415 5457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:34.482048 5458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e0cf459aa654a33a39913ecd6772408. Latest consensus state: current_term: 1 leader_uuid: "7e0cf459aa654a33a39913ecd6772408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0cf459aa654a33a39913ecd6772408" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35547 } } }
I20260811 12:20:34.482462 5458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:34.484474 5471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:34.489984 5471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:34.498831 5471 catalog_manager.cc:1347] Generated new cluster ID: 7a2e4e32ac40436298cee1a68dc929f0
I20260811 12:20:34.498955 5471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:34.550686 5471 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:34.552026 5471 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:34.561350 5471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408: Generated new TSK 0
I20260811 12:20:34.562259 5471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:34.572839 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:35547
--builtin_ntp_servers=127.7.39.20:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:20:34.963624 5488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:34.963904 5488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:34.964016 5488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:34.971306 5488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:34.971418 5488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:34.971534 5488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:20:34.980821 5488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:34.982172 5488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:34.984202 5488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:34.997696 5502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:34.997757 5497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:34.997737 5498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:34.998235 5488 server_base.cc:1034] running on GCE node
I20260811 12:20:34.999112 5488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:35.000149 5488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:35.001368 5488 hybrid_clock.cc:648] HybridClock initialized: now 1786450835001312 us; error 46 us; skew 500 ppm
I20260811 12:20:35.001749 5488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:35.004107 5488 webserver.cc:460] Webserver started at http://127.7.39.1:42325/ using document root <none> and password file <none>
I20260811 12:20:35.004875 5488 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:35.004981 5488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:35.005353 5488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:35.008448 5488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "00d71889129f47ca8a8b5f4c124e6df0"
format_stamp: "Formatted at 2026-08-11 12:20:35 on dist-test-slave-rmhg"
server_key: "ce8ea5ab06e8f22acfc3c4f630602864"
server_key_iv: "4c4d1ec7974170b090dda8d2009de71a"
server_key_version: "encryptionkey@0"
I20260811 12:20:35.009369 5488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "00d71889129f47ca8a8b5f4c124e6df0"
format_stamp: "Formatted at 2026-08-11 12:20:35 on dist-test-slave-rmhg"
server_key: "ce8ea5ab06e8f22acfc3c4f630602864"
server_key_iv: "4c4d1ec7974170b090dda8d2009de71a"
server_key_version: "encryptionkey@0"
I20260811 12:20:35.015046 5488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:20:35.018926 5512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:35.020499 5488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:20:35.020697 5488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "00d71889129f47ca8a8b5f4c124e6df0"
format_stamp: "Formatted at 2026-08-11 12:20:35 on dist-test-slave-rmhg"
server_key: "ce8ea5ab06e8f22acfc3c4f630602864"
server_key_iv: "4c4d1ec7974170b090dda8d2009de71a"
server_key_version: "encryptionkey@0"
I20260811 12:20:35.020927 5488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:35.049985 5488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:35.050866 5488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:35.051115 5488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:35.052192 5488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:35.053788 5488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:35.053882 5488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:35.053982 5488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:35.054044 5488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:35.085592 5488 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:45717
I20260811 12:20:35.085665 5689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:45717 every 8 connection(s)
I20260811 12:20:35.086974 5488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:20:35.092806 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 5488
I20260811 12:20:35.092978 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:20:35.093472 7324 external_mini_cluster.cc:1428] Setting key e4a48f812cc2d800e5e9eedc1a4a024e
I20260811 12:20:35.096222 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:35547
--builtin_ntp_servers=127.7.39.20:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:20:35.100013 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:35.089653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:40741 (local address 127.7.39.62:35547)
0811 12:20:35.090100 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:35.090105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:35.091370 (+ 1265us) server_negotiation.cc:407] Connection header received
0811 12:20:35.093296 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:35.093302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:35.093393 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:35.093532 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:35.094719 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:35.095530 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:35.097230 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:35.097654 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:35.097777 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:35.099153 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:35.099158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:35.099162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:35.099181 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:35.099223 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:35.099235 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:35.099309 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:35.099546 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 12:20:35.099655 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":221,"threads_started":1}
I20260811 12:20:35.101209 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:35.090269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:35547 (local address 127.7.39.1:40741)
0811 12:20:35.091200 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:35.091249 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:35.092935 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:35.093778 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:35.093787 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:35.093887 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:35.094533 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:35.094557 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:35.095686 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:35.095690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:35.097053 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:35.097067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:35.097903 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:35.097911 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:35.098148 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:35.098815 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:35.098818 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:35.098865 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:35.098952 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:35.099324 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:35.099337 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:35.099345 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:20:35.099516 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 12:20:35.100027 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":279,"threads_started":1}
I20260811 12:20:35.102996 5691 heartbeater.cc:347] Connected to a master server at 127.7.39.62:35547
I20260811 12:20:35.103338 5691 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:35.104188 5691 heartbeater.cc:511] Master 127.7.39.62:35547 requested a full tablet report, sending...
I20260811 12:20:35.106874 5395 ts_manager.cc:194] Registered new tserver with Master: 00d71889129f47ca8a8b5f4c124e6df0 (127.7.39.1:45717)
I20260811 12:20:35.108860 5395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.1:40741
W20260811 12:20:35.450903 5697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:35.451203 5697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:35.451289 5697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:35.458673 5697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:35.458798 5697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:35.458926 5697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:20:35.468573 5697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:35.470062 5697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:35.471989 5697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:35.486014 5711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:35.485975 5707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:35.486505 5708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:35.486935 5697 server_base.cc:1034] running on GCE node
I20260811 12:20:35.487634 5697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:35.488709 5697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:35.489950 5697 hybrid_clock.cc:648] HybridClock initialized: now 1786450835489901 us; error 50 us; skew 500 ppm
I20260811 12:20:35.490315 5697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:35.492717 5697 webserver.cc:460] Webserver started at http://127.7.39.2:37029/ using document root <none> and password file <none>
I20260811 12:20:35.493497 5697 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:35.493609 5697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:35.493933 5697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:35.496940 5697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "2b0ad047d48940139e2d111a85385f29"
format_stamp: "Formatted at 2026-08-11 12:20:35 on dist-test-slave-rmhg"
server_key: "78ad97934cd8ac8fdbe623c521c1e999"
server_key_iv: "1558f371edc7774f206970e79a9d9a04"
server_key_version: "encryptionkey@0"
I20260811 12:20:35.497844 5697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "2b0ad047d48940139e2d111a85385f29"
format_stamp: "Formatted at 2026-08-11 12:20:35 on dist-test-slave-rmhg"
server_key: "78ad97934cd8ac8fdbe623c521c1e999"
server_key_iv: "1558f371edc7774f206970e79a9d9a04"
server_key_version: "encryptionkey@0"
I20260811 12:20:35.503350 5697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:20:35.507529 5721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:35.509096 5697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:20:35.509320 5697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "2b0ad047d48940139e2d111a85385f29"
format_stamp: "Formatted at 2026-08-11 12:20:35 on dist-test-slave-rmhg"
server_key: "78ad97934cd8ac8fdbe623c521c1e999"
server_key_iv: "1558f371edc7774f206970e79a9d9a04"
server_key_version: "encryptionkey@0"
I20260811 12:20:35.509604 5697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:35.557303 5697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:35.558241 5697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:35.558516 5697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:35.559563 5697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:35.561173 5697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:35.561281 5697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:35.561406 5697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:35.561472 5697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:35.593611 5697 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:37469
I20260811 12:20:35.593672 5911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:37469 every 8 connection(s)
I20260811 12:20:35.594977 5697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:20:35.602121 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 5697
I20260811 12:20:35.602311 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:20:35.602875 7324 external_mini_cluster.cc:1428] Setting key 5287bdb966f286a5f1cc09ef0bebc3b3
I20260811 12:20:35.605939 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:35547
--builtin_ntp_servers=127.7.39.20:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:20:35.608474 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:35.597646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:42845 (local address 127.7.39.62:35547)
0811 12:20:35.597861 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:35.597867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:35.599340 (+ 1473us) server_negotiation.cc:407] Connection header received
0811 12:20:35.601142 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:35.601146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:35.601248 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:35.601341 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:35.602818 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:35.603597 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:35.605611 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:35.606034 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:35.606162 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:35.607654 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:35.607659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:35.607662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:35.607682 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:35.607716 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:35.607728 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:35.607804 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:35.608030 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 12:20:35.608147 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:20:35.609586 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:35.598220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:35547 (local address 127.7.39.2:42845)
0811 12:20:35.599173 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:35.599221 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:35.600887 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:35.601596 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:35.601605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:35.601727 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:35.602615 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:35.602643 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:35.603756 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:35.603762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:35.605453 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:35.605470 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:35.606396 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:35.606401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:35.606627 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:35.607310 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:35.607313 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:35.607358 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:35.607447 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:35.607818 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:35.607830 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:35.607838 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:20:35.608004 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 12:20:35.608450 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":288,"threads_started":1}
I20260811 12:20:35.610970 5912 heartbeater.cc:347] Connected to a master server at 127.7.39.62:35547
I20260811 12:20:35.611322 5912 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:35.612262 5912 heartbeater.cc:511] Master 127.7.39.62:35547 requested a full tablet report, sending...
I20260811 12:20:35.613934 5395 ts_manager.cc:194] Registered new tserver with Master: 2b0ad047d48940139e2d111a85385f29 (127.7.39.2:37469)
I20260811 12:20:35.614856 5395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.2:42845
W20260811 12:20:35.960604 5915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:35.960916 5915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:35.961032 5915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:35.968530 5915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:35.968644 5915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:35.968788 5915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:20:35.978106 5915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:35.979605 5915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:35.981621 5915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:35.995723 5922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:35.995761 5926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:35.995656 5923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:35.996376 5915 server_base.cc:1034] running on GCE node
I20260811 12:20:35.997092 5915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:35.998122 5915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:35.999364 5915 hybrid_clock.cc:648] HybridClock initialized: now 1786450835999300 us; error 44 us; skew 500 ppm
I20260811 12:20:35.999739 5915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:36.002140 5915 webserver.cc:460] Webserver started at http://127.7.39.3:40787/ using document root <none> and password file <none>
I20260811 12:20:36.002853 5915 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:36.002962 5915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:36.003284 5915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:36.006378 5915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "933811dd12d34098b3b129d3fac498c6"
format_stamp: "Formatted at 2026-08-11 12:20:36 on dist-test-slave-rmhg"
server_key: "cc07159bd6b479fadf39825cdc7875d2"
server_key_iv: "468487bd97b5fa3f05765d949e04db7a"
server_key_version: "encryptionkey@0"
I20260811 12:20:36.007232 5915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "933811dd12d34098b3b129d3fac498c6"
format_stamp: "Formatted at 2026-08-11 12:20:36 on dist-test-slave-rmhg"
server_key: "cc07159bd6b479fadf39825cdc7875d2"
server_key_iv: "468487bd97b5fa3f05765d949e04db7a"
server_key_version: "encryptionkey@0"
I20260811 12:20:36.012581 5915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 12:20:36.016167 5940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:36.017545 5915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:20:36.017688 5915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "933811dd12d34098b3b129d3fac498c6"
format_stamp: "Formatted at 2026-08-11 12:20:36 on dist-test-slave-rmhg"
server_key: "cc07159bd6b479fadf39825cdc7875d2"
server_key_iv: "468487bd97b5fa3f05765d949e04db7a"
server_key_version: "encryptionkey@0"
I20260811 12:20:36.017886 5915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:36.050812 5915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:36.051716 5915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:36.051994 5915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:36.053045 5915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:36.054693 5915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:36.054785 5915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:36.054889 5915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:36.054952 5915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:36.087296 5915 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:39845
I20260811 12:20:36.087411 6111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:39845 every 8 connection(s)
I20260811 12:20:36.088627 5915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:20:36.091393 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 5915
I20260811 12:20:36.091575 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:20:36.092046 7324 external_mini_cluster.cc:1428] Setting key e62d3fb1fc9e53d0f513a876f6525ff8
I20260811 12:20:36.102399 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.091277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:43953 (local address 127.7.39.62:35547)
0811 12:20:36.091501 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.091507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.093448 (+ 1941us) server_negotiation.cc:407] Connection header received
0811 12:20:36.095759 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.095764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.095851 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.095949 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:36.097205 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.097957 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.099582 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.100038 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.100153 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.101678 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:36.101684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:36.101687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:36.101708 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:36.101745 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:36.101758 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:36.101840 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.102032 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.102124 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:20:36.103614 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.091869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:35547 (local address 127.7.39.3:43953)
0811 12:20:36.093290 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:36.093346 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:36.095469 (+ 2123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:36.096198 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:36.096207 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:36.096310 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:36.097009 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.097029 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.098105 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.098108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.099410 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.099423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.100289 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.100296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.100542 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.101304 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:36.101306 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:36.101351 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:36.101442 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:36.101845 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:36.101852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:36.101857 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:20:36.102012 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 12:20:36.102382 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":606,"threads_started":1}
I20260811 12:20:36.105644 6112 heartbeater.cc:347] Connected to a master server at 127.7.39.62:35547
I20260811 12:20:36.106098 6112 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:36.107008 6112 heartbeater.cc:511] Master 127.7.39.62:35547 requested a full tablet report, sending...
I20260811 12:20:36.108600 5395 ts_manager.cc:194] Registered new tserver with Master: 933811dd12d34098b3b129d3fac498c6 (127.7.39.3:39845)
I20260811 12:20:36.109489 5395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.3:43953
I20260811 12:20:36.112605 5691 heartbeater.cc:503] Master 127.7.39.62:35547 was elected leader, sending a full tablet report...
I20260811 12:20:36.120544 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:20:36.131438 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.125145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.7.39.62:35547)
0811 12:20:36.125382 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.125389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.125859 (+ 470us) server_negotiation.cc:407] Connection header received
0811 12:20:36.126032 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.126038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.126112 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.126199 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:36.126756 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.127553 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.128952 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.129390 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.129514 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.130708 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:36.130713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:36.130716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:36.130737 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:36.130771 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:36.130783 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:36.130913 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.131106 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.131163 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:20:36.135751 5395 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:20:36.142832 5395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:20:36.168355 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.156154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:39845 (local address 127.0.0.1:60468)
0811 12:20:36.159461 (+ 3307us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:36.159508 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:36.159707 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:36.160877 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:36.160882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:36.160934 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:36.161531 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.161551 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.163055 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.163060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.165468 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.165483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.167262 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.167269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.167479 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.167541 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 12:20:36.167676 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 12:20:36.167913 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2966,"mutex_wait_us":3,"spinlock_wait_cycles":8448,"thread_start_us":2652,"threads_started":1}
I20260811 12:20:36.168788 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.156612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.7.39.2:37469)
0811 12:20:36.157559 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.157569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.159635 (+ 2066us) server_negotiation.cc:407] Connection header received
0811 12:20:36.159992 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.160001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.160273 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.160461 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:36.161576 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.163202 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.166315 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.167407 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.167543 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.167800 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.168122 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.168315 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":593,"threads_started":1}
I20260811 12:20:36.169224 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.156406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.7.39.3:39845)
0811 12:20:36.158072 (+ 1666us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.158080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.160298 (+ 2218us) server_negotiation.cc:407] Connection header received
0811 12:20:36.160420 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.160427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.160664 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.160874 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:36.161758 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.162866 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.165682 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.167019 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.167181 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.167485 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.168675 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.168869 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"thread_start_us":1338,"threads_started":1}
I20260811 12:20:36.169425 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.158520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.7.39.1:45717)
0811 12:20:36.160681 (+ 2161us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.160690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.160719 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:20:36.160800 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.160807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.161044 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.161281 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:36.161933 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.163402 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.166220 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.167685 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.168477 (+ 792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.168778 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.168858 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.169045 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1971,"thread_start_us":1818,"threads_started":1}
I20260811 12:20:36.169986 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.156210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37469 (local address 127.0.0.1:58846)
0811 12:20:36.159458 (+ 3248us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:36.159508 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:36.159746 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:36.160720 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:36.160727 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:36.160759 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:36.161268 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.161282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.163375 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.163380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.166079 (+ 2699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.166090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.167654 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.167663 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.167875 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.167913 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:20:36.169196 (+ 1283us) client_negotiation.cc:240] Negotiation successful
0811 12:20:36.169666 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3116,"mutex_wait_us":252,"thread_start_us":2076,"threads_started":1}
I20260811 12:20:36.170269 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.158169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:45717 (local address 127.0.0.1:57442)
0811 12:20:36.160059 (+ 1890us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:36.160078 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:36.160175 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:36.161284 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:36.161292 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:36.161317 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:36.161743 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.161756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.163591 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.163597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.165994 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.166010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.167908 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.167914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.168085 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.168113 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 12:20:36.168176 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 12:20:36.168311 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1748,"mutex_wait_us":68,"thread_start_us":1026,"threads_started":1}
I20260811 12:20:36.173521 5587 tablet_service.cc:1467] Processing CreateTablet for tablet f06562689f2848b099bde96d13aa94cb (DEFAULT_TABLE table=test-table [id=7a96bb4284094216af71d4e8f9aab030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:36.173614 6008 tablet_service.cc:1467] Processing CreateTablet for tablet f06562689f2848b099bde96d13aa94cb (DEFAULT_TABLE table=test-table [id=7a96bb4284094216af71d4e8f9aab030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:36.173831 5809 tablet_service.cc:1467] Processing CreateTablet for tablet f06562689f2848b099bde96d13aa94cb (DEFAULT_TABLE table=test-table [id=7a96bb4284094216af71d4e8f9aab030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:36.175175 5587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06562689f2848b099bde96d13aa94cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:36.175202 6008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06562689f2848b099bde96d13aa94cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:36.175523 5809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06562689f2848b099bde96d13aa94cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:36.188045 6136 tablet_bootstrap.cc:492] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Bootstrap starting.
I20260811 12:20:36.191505 6137 tablet_bootstrap.cc:492] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Bootstrap starting.
I20260811 12:20:36.193670 6136 tablet_bootstrap.cc:654] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:36.193959 6139 tablet_bootstrap.cc:492] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Bootstrap starting.
I20260811 12:20:36.195350 6136 log.cc:824] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:36.195685 6137 tablet_bootstrap.cc:654] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:36.196904 6137 log.cc:824] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:36.197589 6139 tablet_bootstrap.cc:654] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:36.198909 6139 log.cc:824] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:36.198988 6136 tablet_bootstrap.cc:492] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: No bootstrap required, opened a new log
I20260811 12:20:36.199316 6136 ts_tablet_manager.cc:1397] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260811 12:20:36.201331 6137 tablet_bootstrap.cc:492] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: No bootstrap required, opened a new log
I20260811 12:20:36.201661 6137 ts_tablet_manager.cc:1397] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260811 12:20:36.207912 6139 tablet_bootstrap.cc:492] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: No bootstrap required, opened a new log
I20260811 12:20:36.208235 6139 ts_tablet_manager.cc:1397] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260811 12:20:36.210158 6136 raft_consensus.cc:348] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.210714 6136 raft_consensus.cc:374] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:36.210832 6136 raft_consensus.cc:729] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d71889129f47ca8a8b5f4c124e6df0, State: Initialized, Role: FOLLOWER
I20260811 12:20:36.211691 6136 consensus_queue.cc:260] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.213984 6136 ts_tablet_manager.cc:1428] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 12:20:36.214599 6137 raft_consensus.cc:348] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.215076 6137 raft_consensus.cc:374] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:36.215195 6137 raft_consensus.cc:729] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b0ad047d48940139e2d111a85385f29, State: Initialized, Role: FOLLOWER
I20260811 12:20:36.215875 6137 consensus_queue.cc:260] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.217100 5912 heartbeater.cc:503] Master 127.7.39.62:35547 was elected leader, sending a full tablet report...
I20260811 12:20:36.217882 6137 ts_tablet_manager.cc:1428] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260811 12:20:36.218492 6139 raft_consensus.cc:348] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.218993 6139 raft_consensus.cc:374] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:36.219125 6139 raft_consensus.cc:729] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933811dd12d34098b3b129d3fac498c6, State: Initialized, Role: FOLLOWER
I20260811 12:20:36.219808 6139 consensus_queue.cc:260] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.221359 6112 heartbeater.cc:503] Master 127.7.39.62:35547 was elected leader, sending a full tablet report...
I20260811 12:20:36.221899 6139 ts_tablet_manager.cc:1428] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
W20260811 12:20:36.339648 5693 tablet.cc:2369] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:20:36.340400 6113 tablet.cc:2369] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:36.342295 6144 raft_consensus.cc:482] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:36.342597 6144 raft_consensus.cc:504] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.344460 6144 leader_election.cc:288] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 933811dd12d34098b3b129d3fac498c6 (127.7.39.3:39845), 2b0ad047d48940139e2d111a85385f29 (127.7.39.2:37469)
W20260811 12:20:36.347071 5913 tablet.cc:2369] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:36.354159 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.345018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:40607 (local address 127.7.39.2:37469)
0811 12:20:36.345297 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.345305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.345888 (+ 583us) server_negotiation.cc:407] Connection header received
0811 12:20:36.346095 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.346102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.346199 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.346306 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:36.347392 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.348537 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.351817 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.353126 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.353322 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.353425 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.353749 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.353892 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:20:36.354185 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.344788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:37469 (local address 127.7.39.1:40607)
0811 12:20:36.345677 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:36.345700 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:36.345910 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:36.346385 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:36.346390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:36.346462 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:36.347182 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.347192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.348750 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.348757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.351482 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.351508 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.353397 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.353404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.353549 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.353581 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:20:36.353667 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:20:36.353784 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":140,"spinlock_wait_cycles":57216,"thread_start_us":527,"threads_started":1}
I20260811 12:20:36.354344 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.345039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:32807 (local address 127.7.39.3:39845)
0811 12:20:36.345273 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.345280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.345912 (+ 632us) server_negotiation.cc:407] Connection header received
0811 12:20:36.346095 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.346100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.346188 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.346306 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:36.347394 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.348584 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.351824 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.353230 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.353348 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.353442 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.353982 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.354121 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:20:36.355026 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.344788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:39845 (local address 127.7.39.1:32807)
0811 12:20:36.345763 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:36.345779 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:36.345952 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:36.346384 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:36.346390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:36.346442 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:36.347157 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.347181 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.348750 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.348757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.351482 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:36.351508 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.353577 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.353583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:36.353761 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.353795 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:20:36.354529 (+ 734us) client_negotiation.cc:240] Negotiation successful
0811 12:20:36.354679 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":291,"spinlock_wait_cycles":29440,"thread_start_us":632,"threads_started":1}
I20260811 12:20:36.355397 5843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f06562689f2848b099bde96d13aa94cb" candidate_uuid: "00d71889129f47ca8a8b5f4c124e6df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b0ad047d48940139e2d111a85385f29" is_pre_election: true
I20260811 12:20:36.356040 5843 raft_consensus.cc:2454] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d71889129f47ca8a8b5f4c124e6df0 in term 0.
I20260811 12:20:36.356113 6039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f06562689f2848b099bde96d13aa94cb" candidate_uuid: "00d71889129f47ca8a8b5f4c124e6df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933811dd12d34098b3b129d3fac498c6" is_pre_election: true
I20260811 12:20:36.356585 6039 raft_consensus.cc:2454] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d71889129f47ca8a8b5f4c124e6df0 in term 0.
I20260811 12:20:36.356848 5515 leader_election.cc:302] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d71889129f47ca8a8b5f4c124e6df0, 2b0ad047d48940139e2d111a85385f29; no voters:
I20260811 12:20:36.357371 6144 raft_consensus.cc:2781] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:20:36.357528 6144 raft_consensus.cc:482] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:20:36.357635 6144 raft_consensus.cc:3037] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:36.359855 6144 raft_consensus.cc:504] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.360586 6144 leader_election.cc:288] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [CANDIDATE]: Term 1 election: Requested vote from peers 933811dd12d34098b3b129d3fac498c6 (127.7.39.3:39845), 2b0ad047d48940139e2d111a85385f29 (127.7.39.2:37469)
I20260811 12:20:36.361127 6039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f06562689f2848b099bde96d13aa94cb" candidate_uuid: "00d71889129f47ca8a8b5f4c124e6df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933811dd12d34098b3b129d3fac498c6"
I20260811 12:20:36.361207 5843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f06562689f2848b099bde96d13aa94cb" candidate_uuid: "00d71889129f47ca8a8b5f4c124e6df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b0ad047d48940139e2d111a85385f29"
I20260811 12:20:36.361361 6039 raft_consensus.cc:3037] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:36.361420 5843 raft_consensus.cc:3037] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:36.363624 6039 raft_consensus.cc:2454] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d71889129f47ca8a8b5f4c124e6df0 in term 1.
I20260811 12:20:36.363778 5843 raft_consensus.cc:2454] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d71889129f47ca8a8b5f4c124e6df0 in term 1.
I20260811 12:20:36.364087 5513 leader_election.cc:302] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d71889129f47ca8a8b5f4c124e6df0, 933811dd12d34098b3b129d3fac498c6; no voters:
I20260811 12:20:36.364372 6144 raft_consensus.cc:2781] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:36.364878 6144 raft_consensus.cc:686] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 1 LEADER]: Becoming Leader. State: Replica: 00d71889129f47ca8a8b5f4c124e6df0, State: Running, Role: LEADER
I20260811 12:20:36.365476 6144 consensus_queue.cc:237] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } }
I20260811 12:20:36.370193 5393 catalog_manager.cc:5495] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00d71889129f47ca8a8b5f4c124e6df0 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "00d71889129f47ca8a8b5f4c124e6df0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d71889129f47ca8a8b5f4c124e6df0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 45717 } health_report { overall_health: HEALTHY } } }
I20260811 12:20:36.412382 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.406691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.7.39.1:45717)
0811 12:20:36.406920 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.406927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.406947 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:20:36.407038 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.407041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.407142 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.407281 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:20:36.407703 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.408430 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.410008 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.410443 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.410559 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.411238 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:20:36.411757 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:20:36.411942 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.412113 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.412160 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:20:36.420624 6039 raft_consensus.cc:1261] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 1 FOLLOWER]: Refusing update from remote peer 00d71889129f47ca8a8b5f4c124e6df0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:36.420707 5843 raft_consensus.cc:1261] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 1 FOLLOWER]: Refusing update from remote peer 00d71889129f47ca8a8b5f4c124e6df0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:36.421619 6144 consensus_queue.cc:1035] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "933811dd12d34098b3b129d3fac498c6" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 39845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:36.421981 6152 consensus_queue.cc:1035] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b0ad047d48940139e2d111a85385f29" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 37469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:36.435880 6156 mvcc.cc:205] Tried to move back new op lower bound from 7317302625969393664 to 7317302625757081600. Current Snapshot: MvccSnapshot[applied={T|T < 7317302625969393664}]
I20260811 12:20:36.439849 6155 mvcc.cc:205] Tried to move back new op lower bound from 7317302625969393664 to 7317302625757081600. Current Snapshot: MvccSnapshot[applied={T|T < 7317302625969393664}]
I20260811 12:20:36.442160 6158 mvcc.cc:205] Tried to move back new op lower bound from 7317302625969393664 to 7317302625757081600. Current Snapshot: MvccSnapshot[applied={T|T < 7317302625969393664}]
I20260811 12:20:36.453262 5393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:36.453567 5393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:36.457187 5393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e0cf459aa654a33a39913ecd6772408: Sending DeleteTablet for 3 replicas of tablet f06562689f2848b099bde96d13aa94cb
I20260811 12:20:36.458706 5587 tablet_service.cc:1514] Processing DeleteTablet for tablet f06562689f2848b099bde96d13aa94cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:36 UTC) from {username='slave'} at 127.0.0.1:57442
I20260811 12:20:36.458683 6008 tablet_service.cc:1514] Processing DeleteTablet for tablet f06562689f2848b099bde96d13aa94cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:36 UTC) from {username='slave'} at 127.0.0.1:60468
I20260811 12:20:36.459455 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet f06562689f2848b099bde96d13aa94cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:36 UTC) from {username='slave'} at 127.0.0.1:58846
I20260811 12:20:36.459733 6171 tablet_replica.cc:330] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: stopping tablet replica
I20260811 12:20:36.460289 6171 raft_consensus.cc:2229] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:20:36.460464 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 5488
I20260811 12:20:36.461185 6171 raft_consensus.cc:2258] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:36.463740 6171 ts_tablet_manager.cc:1905] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:36.472816 6171 ts_tablet_manager.cc:1918] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:36.473237 6171 log.cc:1196] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/f06562689f2848b099bde96d13aa94cb
I20260811 12:20:36.473987 6171 ts_tablet_manager.cc:1939] T f06562689f2848b099bde96d13aa94cb P 00d71889129f47ca8a8b5f4c124e6df0: Deleting consensus metadata
I20260811 12:20:36.476502 5372 catalog_manager.cc:4841] TS 00d71889129f47ca8a8b5f4c124e6df0 (127.7.39.1:45717): tablet f06562689f2848b099bde96d13aa94cb (table test-table [id=7a96bb4284094216af71d4e8f9aab030]) successfully deleted
I20260811 12:20:36.478473 6175 tablet_replica.cc:330] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: stopping tablet replica
I20260811 12:20:36.479104 6175 raft_consensus.cc:2229] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:36.479818 6173 tablet_replica.cc:330] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: stopping tablet replica
I20260811 12:20:36.480280 6173 raft_consensus.cc:2229] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:36.480829 6173 raft_consensus.cc:2258] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:36.481587 6175 raft_consensus.cc:2258] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:36.483757 6173 ts_tablet_manager.cc:1905] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:36.483774 6175 ts_tablet_manager.cc:1905] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:36.487383 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.463153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.7.39.1:45717)
0811 12:20:36.463333 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.463339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.477789 (+ 14450us) server_negotiation.cc:407] Connection header received
0811 12:20:36.477978 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.477984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.478098 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.478208 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:36.479031 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.480431 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.482921 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.483575 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.483681 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.485404 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:36.485416 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:36.485435 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:36.486412 (+ 977us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:36.486478 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:36.486507 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:36.486608 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.486889 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.487004 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 12:20:36.488632 5683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:36.490978 6175 ts_tablet_manager.cc:1918] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:36.491168 6175 log.cc:1196] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/f06562689f2848b099bde96d13aa94cb
I20260811 12:20:36.491750 6175 ts_tablet_manager.cc:1939] T f06562689f2848b099bde96d13aa94cb P 2b0ad047d48940139e2d111a85385f29: Deleting consensus metadata
I20260811 12:20:36.491820 6173 ts_tablet_manager.cc:1918] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:36.492007 6173 log.cc:1196] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/f06562689f2848b099bde96d13aa94cb
I20260811 12:20:36.492513 6173 ts_tablet_manager.cc:1939] T f06562689f2848b099bde96d13aa94cb P 933811dd12d34098b3b129d3fac498c6: Deleting consensus metadata
I20260811 12:20:36.493553 5373 catalog_manager.cc:4841] TS 2b0ad047d48940139e2d111a85385f29 (127.7.39.2:37469): tablet f06562689f2848b099bde96d13aa94cb (table test-table [id=7a96bb4284094216af71d4e8f9aab030]) successfully deleted
I20260811 12:20:36.494361 5371 catalog_manager.cc:4841] TS 933811dd12d34098b3b129d3fac498c6 (127.7.39.3:39845): tablet f06562689f2848b099bde96d13aa94cb (table test-table [id=7a96bb4284094216af71d4e8f9aab030]) successfully deleted
I20260811 12:20:36.601500 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 5488
I20260811 12:20:36.627638 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 5697
I20260811 12:20:36.640679 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.628527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58850 (local address 127.7.39.2:37469)
0811 12:20:36.628744 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.628750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.628771 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:20:36.628960 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.628965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.629088 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.629213 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:36.629631 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.630413 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.631729 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.632149 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.632260 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.639098 (+ 6838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:36.639111 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:36.639129 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:36.639799 (+ 670us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:36.639862 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:36.639889 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:36.640036 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.640194 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.640330 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:20:36.641474 5906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:36.732219 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 5697
I20260811 12:20:36.753022 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 5915
I20260811 12:20:36.760196 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:36.754014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60484 (local address 127.7.39.3:39845)
0811 12:20:36.754217 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:36.754223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:36.754244 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:20:36.754368 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:36.754373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:36.754463 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:36.754554 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:36.754984 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.755706 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.757025 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:36.757609 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:36.757726 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:36.758820 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:36.758831 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:36.758847 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:36.759437 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:36.759486 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:36.759505 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:36.759606 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:36.759749 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 12:20:36.759869 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:20:36.760898 6107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:36.850467 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 5915
I20260811 12:20:36.871917 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 5349
I20260811 12:20:36.873124 5442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:36.984833 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 5349
2026-08-11T12:20:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3124 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:37 dist-test-slave-rmhg krb5kdc[6186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:37 dist-test-slave-rmhg krb5kdc[6186](info): set up 2 sockets
Aug 11 12:20:37 dist-test-slave-rmhg krb5kdc[6186](info): commencing operation
krb5kdc: starting...
W20260811 12:20:38.260946 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:38 dist-test-slave-rmhg krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:20:38.603541 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:33407
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:33407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:38.965411 6205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:38.965744 6205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:38.965890 6205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:38.973122 6205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:38.973281 6205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:38.973366 6205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:38.973450 6205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:38.973522 6205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:38.983848 6205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:33407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:33407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:38.985399 6205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:38.986822 6205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:38.998989 6213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:38.999096 6214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:38.999068 6217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:38.999935 6205 server_base.cc:1034] running on GCE node
I20260811 12:20:39.000730 6205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:39.002245 6205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:39.003512 6205 hybrid_clock.cc:648] HybridClock initialized: now 1786450839003445 us; error 51 us; skew 500 ppm
I20260811 12:20:39.003868 6205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:39 dist-test-slave-rmhg krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450839, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:39.007336 6205 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:20:39.008870 6205 webserver.cc:460] Webserver started at http://127.7.39.62:37219/ using document root <none> and password file <none>
I20260811 12:20:39.009649 6205 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:39.009779 6205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:39.010141 6205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:39.013326 6205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "ac0ddc7f26a348629cecd4719dccb783"
format_stamp: "Formatted at 2026-08-11 12:20:39 on dist-test-slave-rmhg"
server_key: "0ac27fa9a24097880a5568c058c31aa4"
server_key_iv: "306b88bbcb79f6509ad01b5f485f30a4"
server_key_version: "encryptionkey@0"
I20260811 12:20:39.014101 6205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "ac0ddc7f26a348629cecd4719dccb783"
format_stamp: "Formatted at 2026-08-11 12:20:39 on dist-test-slave-rmhg"
server_key: "0ac27fa9a24097880a5568c058c31aa4"
server_key_iv: "306b88bbcb79f6509ad01b5f485f30a4"
server_key_version: "encryptionkey@0"
I20260811 12:20:39.019594 6205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:20:39.023417 6230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:39.024972 6205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:20:39.025213 6205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "ac0ddc7f26a348629cecd4719dccb783"
format_stamp: "Formatted at 2026-08-11 12:20:39 on dist-test-slave-rmhg"
server_key: "0ac27fa9a24097880a5568c058c31aa4"
server_key_iv: "306b88bbcb79f6509ad01b5f485f30a4"
server_key_version: "encryptionkey@0"
I20260811 12:20:39.025435 6205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:39.047261 6205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:39.051863 6205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:39.052155 6205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:39.070266 6205 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:33407
I20260811 12:20:39.070292 6313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:33407 every 8 connection(s)
I20260811 12:20:39.071772 6205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:39.075765 6316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:39.076987 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 6205
I20260811 12:20:39.077210 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:39.077699 7324 external_mini_cluster.cc:1428] Setting key 20e85583886abda2207f42ea72e9308e
I20260811 12:20:39.090404 6316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783: Bootstrap starting.
Aug 11 12:20:39 dist-test-slave-rmhg krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:39.094765 6316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:39.095971 6316 log.cc:824] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:39.099625 6316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783: No bootstrap required, opened a new log
I20260811 12:20:39.103142 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:39.079643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.7.39.62:33407)
0811 12:20:39.080461 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:39.080473 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:39.080520 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:20:39.082250 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:39.082276 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:39.082716 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:39.083081 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:39.085001 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:39.086410 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:39.088331 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:39.089075 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:39.089293 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:39.094644 (+ 5351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:39.094657 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:39.094675 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:39.094727 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:39.098087 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:39.099143 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:39.099151 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:39.099159 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:39.099356 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:39.099864 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:39.099871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:39.099874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:39.100516 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:39.100789 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:39.101376 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 12:20:39.101894 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":380,"threads_started":1}
I20260811 12:20:39.108774 6316 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0ddc7f26a348629cecd4719dccb783" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33407 } }
I20260811 12:20:39.109303 6316 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:39.109433 6316 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac0ddc7f26a348629cecd4719dccb783, State: Initialized, Role: FOLLOWER
I20260811 12:20:39.110145 6316 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0ddc7f26a348629cecd4719dccb783" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33407 } }
I20260811 12:20:39.110414 6316 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:39.110559 6316 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:39.110735 6316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:39.113705 6316 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0ddc7f26a348629cecd4719dccb783" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33407 } }
I20260811 12:20:39.114284 6316 leader_election.cc:302] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac0ddc7f26a348629cecd4719dccb783; no voters:
I20260811 12:20:39.114931 6316 leader_election.cc:288] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:39.115082 6322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:39.115808 6322 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [term 1 LEADER]: Becoming Leader. State: Replica: ac0ddc7f26a348629cecd4719dccb783, State: Running, Role: LEADER
I20260811 12:20:39.116451 6322 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0ddc7f26a348629cecd4719dccb783" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33407 } }
I20260811 12:20:39.117569 6316 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:39.119695 6325 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac0ddc7f26a348629cecd4719dccb783" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0ddc7f26a348629cecd4719dccb783" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33407 } } }
I20260811 12:20:39.119731 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac0ddc7f26a348629cecd4719dccb783. Latest consensus state: current_term: 1 leader_uuid: "ac0ddc7f26a348629cecd4719dccb783" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0ddc7f26a348629cecd4719dccb783" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33407 } } }
I20260811 12:20:39.120134 6325 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:39.120252 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:39.122505 6329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:39.129060 6329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:39.141010 6329 catalog_manager.cc:1347] Generated new cluster ID: fa362c37e77c4119bd9b67d5211b8221
I20260811 12:20:39.141197 6329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:39.163478 6329 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:39.164930 6329 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:39.175148 6329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783: Generated new TSK 0
I20260811 12:20:39.176105 6329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.39.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
Entry for principal kudu/127.7.39.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab.
I20260811 12:20:39.258496 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:33407
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:39.616209 6352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:39.616523 6352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:39.616623 6352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:39.624301 6352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:39.624456 6352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:39.624629 6352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:20:39.635990 6352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.1.keytab
--principal=kudu/127.7.39.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:33407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:39.637609 6352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:39.639037 6352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:39.652699 6357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:39.652669 6358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:39.652740 6361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:39.653568 6352 server_base.cc:1034] running on GCE node
I20260811 12:20:39.654206 6352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:39.655187 6352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:39.656427 6352 hybrid_clock.cc:648] HybridClock initialized: now 1786450839656370 us; error 41 us; skew 500 ppm
I20260811 12:20:39.656803 6352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:39 dist-test-slave-rmhg krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450839, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:39.660307 6352 init.cc:377] Logged in from keytab as kudu/127.7.39.1@KRBTEST.COM (short username kudu)
I20260811 12:20:39.662078 6352 webserver.cc:460] Webserver started at http://127.7.39.1:39381/ using document root <none> and password file <none>
I20260811 12:20:39.662850 6352 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:39.662961 6352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:39.663273 6352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:39.666520 6352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "31dc1247f94b499e9044043c0f7c02f0"
format_stamp: "Formatted at 2026-08-11 12:20:39 on dist-test-slave-rmhg"
server_key: "fd6f27b20323ccbf5e8d74ffe518f096"
server_key_iv: "b3d571338fa4a5afbfd3dc162844193f"
server_key_version: "encryptionkey@0"
I20260811 12:20:39.667367 6352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "31dc1247f94b499e9044043c0f7c02f0"
format_stamp: "Formatted at 2026-08-11 12:20:39 on dist-test-slave-rmhg"
server_key: "fd6f27b20323ccbf5e8d74ffe518f096"
server_key_iv: "b3d571338fa4a5afbfd3dc162844193f"
server_key_version: "encryptionkey@0"
I20260811 12:20:39.673087 6352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:20:39.677053 6375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:39.678630 6352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:39.678804 6352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "31dc1247f94b499e9044043c0f7c02f0"
format_stamp: "Formatted at 2026-08-11 12:20:39 on dist-test-slave-rmhg"
server_key: "fd6f27b20323ccbf5e8d74ffe518f096"
server_key_iv: "b3d571338fa4a5afbfd3dc162844193f"
server_key_version: "encryptionkey@0"
I20260811 12:20:39.679018 6352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:39.711230 6352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:39.715781 6352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:39.716079 6352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:39.717226 6352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:39.718788 6352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:39.718866 6352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:39.718986 6352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:39.719049 6352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:39.751948 6352 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:33141
I20260811 12:20:39.751999 6569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:33141 every 8 connection(s)
I20260811 12:20:39.753412 6352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:20:39.763098 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 6352
I20260811 12:20:39.763264 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:20:39.763690 7324 external_mini_cluster.cc:1428] Setting key d7450d982909e69574a75ed5cf32dabc
Aug 11 12:20:39 dist-test-slave-rmhg krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450839, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.1@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:39.774006 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:39.756050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:40453 (local address 127.7.39.62:33407)
0811 12:20:39.756576 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:39.756583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:39.757678 (+ 1095us) server_negotiation.cc:407] Connection header received
0811 12:20:39.759558 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:39.759563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:39.759637 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:39.759741 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:39.761413 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:39.762164 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:39.763993 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:39.764470 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:39.764604 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:39.768384 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:39.768389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:39.768392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:39.768426 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:39.770869 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:39.771614 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:39.771618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:39.771620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:39.771750 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:39.772194 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:39.772198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:39.772200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:39.772470 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:39.772615 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:39.773389 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 12:20:39.773651 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":219,"threads_started":1}
I20260811 12:20:39.775162 6576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:39.756671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:33407 (local address 127.7.39.1:40453)
0811 12:20:39.757497 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:39.757548 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:39.759271 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:39.759955 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:39.759963 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:39.760540 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:39.761209 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:39.761238 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:39.762356 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:39.762361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:39.763838 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:39.763853 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:39.764710 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:39.764719 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:39.765036 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:39.765866 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:39.765898 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:39.768155 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:39.771011 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:39.771022 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:39.771048 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:39.771479 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:39.771858 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:39.771863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:39.771866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:39.772075 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:39.772602 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:39.772610 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:39.772964 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 12:20:39.773267 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 12:20:39.773863 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":331,"threads_started":1}
I20260811 12:20:39.776856 6570 heartbeater.cc:347] Connected to a master server at 127.7.39.62:33407
I20260811 12:20:39.777211 6570 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:39.777964 6570 heartbeater.cc:511] Master 127.7.39.62:33407 requested a full tablet report, sending...
I20260811 12:20:39.780117 6259 ts_manager.cc:194] Registered new tserver with Master: 31dc1247f94b499e9044043c0f7c02f0 (127.7.39.1:33141)
I20260811 12:20:39.781690 6259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.1@KRBTEST.COM'} at 127.7.39.1:40453
WARNING: no policy specified for kudu/127.7.39.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
Entry for principal kudu/127.7.39.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab.
I20260811 12:20:39.836962 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:33407
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:40.193476 6579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:40.193801 6579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:40.193943 6579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:40.201601 6579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:40.201749 6579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:40.201905 6579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:20:40.212154 6579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.2.keytab
--principal=kudu/127.7.39.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:33407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:40.213677 6579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:40.215091 6579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:40.229465 6586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:40.229527 6594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:40.229494 6587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:40.230197 6579 server_base.cc:1034] running on GCE node
I20260811 12:20:40.231056 6579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:40.232095 6579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:40.233310 6579 hybrid_clock.cc:648] HybridClock initialized: now 1786450840233252 us; error 53 us; skew 500 ppm
I20260811 12:20:40.233644 6579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:40 dist-test-slave-rmhg krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450840, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:40.237051 6579 init.cc:377] Logged in from keytab as kudu/127.7.39.2@KRBTEST.COM (short username kudu)
I20260811 12:20:40.238834 6579 webserver.cc:460] Webserver started at http://127.7.39.2:45877/ using document root <none> and password file <none>
I20260811 12:20:40.239614 6579 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:40.239732 6579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:40.240015 6579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:40.243235 6579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "9f6177fc53b6418c94a79c16601a9db5"
format_stamp: "Formatted at 2026-08-11 12:20:40 on dist-test-slave-rmhg"
server_key: "aac26c851d0574de326c1ce750ec4b8d"
server_key_iv: "88596f24c3cc9530dfb0ad494f796f3b"
server_key_version: "encryptionkey@0"
I20260811 12:20:40.244099 6579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "9f6177fc53b6418c94a79c16601a9db5"
format_stamp: "Formatted at 2026-08-11 12:20:40 on dist-test-slave-rmhg"
server_key: "aac26c851d0574de326c1ce750ec4b8d"
server_key_iv: "88596f24c3cc9530dfb0ad494f796f3b"
server_key_version: "encryptionkey@0"
I20260811 12:20:40.250065 6579 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 12:20:40.253978 6607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:40.255586 6579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:40.255771 6579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "9f6177fc53b6418c94a79c16601a9db5"
format_stamp: "Formatted at 2026-08-11 12:20:40 on dist-test-slave-rmhg"
server_key: "aac26c851d0574de326c1ce750ec4b8d"
server_key_iv: "88596f24c3cc9530dfb0ad494f796f3b"
server_key_version: "encryptionkey@0"
I20260811 12:20:40.255981 6579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:40.277419 6579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:40.282238 6579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:40.282541 6579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:40.283656 6579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:40.285107 6579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:40.285221 6579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 12:20:40.285353 6579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:40.285409 6579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:40.317274 6579 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:34661
I20260811 12:20:40.317324 6800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:34661 every 8 connection(s)
I20260811 12:20:40.318754 6579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:20:40.320910 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 6579
I20260811 12:20:40.321110 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:20:40.321630 7324 external_mini_cluster.cc:1428] Setting key 80e846af372f5ef4184636cd7ac661a7
Aug 11 12:20:40 dist-test-slave-rmhg krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450840, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.2@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:40.342190 6809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.321649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:45699 (local address 127.7.39.62:33407)
0811 12:20:40.322170 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:40.322177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:40.323766 (+ 1589us) server_negotiation.cc:407] Connection header received
0811 12:20:40.325938 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:40.325943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:40.326039 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:40.326161 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:40.327949 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.328945 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.330822 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.331410 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.331541 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.335332 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:40.335339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:40.335343 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:40.335390 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:40.338579 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:40.339463 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:40.339467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:40.339470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:40.339623 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:40.340176 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:40.340181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:40.340183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:40.340486 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:40.340624 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:40.341645 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0811 12:20:40.341873 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":238,"threads_started":1}
I20260811 12:20:40.343623 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.322351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:33407 (local address 127.7.39.2:45699)
0811 12:20:40.323431 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:40.323489 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:40.325601 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:40.326404 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:40.326413 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:40.327061 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:40.327728 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.327749 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.329145 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.329194 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.330661 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.330676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.331676 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.331682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.331908 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.332613 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:40.332645 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:40.335031 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:40.338771 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:40.338781 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:40.338798 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:40.339313 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:40.339747 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:40.339752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:40.339755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:40.340026 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:40.340627 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:40.340635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:40.341127 (+ 492us) client_negotiation.cc:769] Sending connection context
0811 12:20:40.341497 (+ 370us) client_negotiation.cc:240] Negotiation successful
0811 12:20:40.342131 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":432,"threads_started":1}
I20260811 12:20:40.345438 6803 heartbeater.cc:347] Connected to a master server at 127.7.39.62:33407
I20260811 12:20:40.345808 6803 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:40.346628 6803 heartbeater.cc:511] Master 127.7.39.62:33407 requested a full tablet report, sending...
I20260811 12:20:40.348394 6259 ts_manager.cc:194] Registered new tserver with Master: 9f6177fc53b6418c94a79c16601a9db5 (127.7.39.2:34661)
I20260811 12:20:40.349323 6259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.2@KRBTEST.COM'} at 127.7.39.2:45699
WARNING: no policy specified for kudu/127.7.39.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
Entry for principal kudu/127.7.39.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab.
I20260811 12:20:40.400383 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:33407
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:40.760241 6815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:40.760651 6815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:40.760819 6815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:40.770881 6815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:40.771019 6815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:40.771142 6815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:20:40.782290 6815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.3.keytab
--principal=kudu/127.7.39.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:33407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:40.783815 6815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:40.785132 6570 heartbeater.cc:503] Master 127.7.39.62:33407 was elected leader, sending a full tablet report...
I20260811 12:20:40.785503 6815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:40.800500 6828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:40.800480 6827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:40.800513 6831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:40.801426 6815 server_base.cc:1034] running on GCE node
I20260811 12:20:40.802170 6815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:40.803311 6815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:40.804534 6815 hybrid_clock.cc:648] HybridClock initialized: now 1786450840804498 us; error 71 us; skew 500 ppm
I20260811 12:20:40.804872 6815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:40 dist-test-slave-rmhg krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450840, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:40.808302 6815 init.cc:377] Logged in from keytab as kudu/127.7.39.3@KRBTEST.COM (short username kudu)
I20260811 12:20:40.809895 6815 webserver.cc:460] Webserver started at http://127.7.39.3:45643/ using document root <none> and password file <none>
I20260811 12:20:40.810626 6815 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:40.810734 6815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:40.811029 6815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:40.814365 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "03d6d0e8820f44ffacf0c56d0462de41"
format_stamp: "Formatted at 2026-08-11 12:20:40 on dist-test-slave-rmhg"
server_key: "ebd9d6ca8f561b2701f7d2cc32af7a69"
server_key_iv: "b28ac3f65e260bcf5ac08d7cc41491f1"
server_key_version: "encryptionkey@0"
I20260811 12:20:40.815220 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "03d6d0e8820f44ffacf0c56d0462de41"
format_stamp: "Formatted at 2026-08-11 12:20:40 on dist-test-slave-rmhg"
server_key: "ebd9d6ca8f561b2701f7d2cc32af7a69"
server_key_iv: "b28ac3f65e260bcf5ac08d7cc41491f1"
server_key_version: "encryptionkey@0"
I20260811 12:20:40.821135 6815 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 12:20:40.825058 6838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:40.826624 6815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:40.826793 6815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "03d6d0e8820f44ffacf0c56d0462de41"
format_stamp: "Formatted at 2026-08-11 12:20:40 on dist-test-slave-rmhg"
server_key: "ebd9d6ca8f561b2701f7d2cc32af7a69"
server_key_iv: "b28ac3f65e260bcf5ac08d7cc41491f1"
server_key_version: "encryptionkey@0"
I20260811 12:20:40.826992 6815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:40.846923 6815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:40.851485 6815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:40.851748 6815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:40.852746 6815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:40.854265 6815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:40.854342 6815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:40.854439 6815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:40.854482 6815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:40.885349 6815 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:45645
I20260811 12:20:40.885390 7022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:45645 every 8 connection(s)
I20260811 12:20:40.886658 6815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:20:40.895162 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 6815
I20260811 12:20:40.895357 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:20:40.895910 7324 external_mini_cluster.cc:1428] Setting key c1f3fce0a57c310d2bddf8e618855043
Aug 11 12:20:40 dist-test-slave-rmhg krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450840, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.3@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:20:40.908093 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.889390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:56291 (local address 127.7.39.62:33407)
0811 12:20:40.889894 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:40.889901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:40.890928 (+ 1027us) server_negotiation.cc:407] Connection header received
0811 12:20:40.892838 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:40.892844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:40.892935 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:40.893036 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:40.894722 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.895885 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.897925 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.898381 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.898482 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.902337 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:40.902343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:40.902346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:40.902384 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:40.905033 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:40.905850 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:40.905855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:40.905857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:40.905988 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:40.906364 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:40.906367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:40.906369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:40.906653 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:40.906774 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:40.907572 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 12:20:40.907781 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":234,"threads_started":1}
I20260811 12:20:40.909104 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.889979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:33407 (local address 127.7.39.3:56291)
0811 12:20:40.890747 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:40.890792 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:40.892538 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:40.893336 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:40.893344 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:40.893901 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:40.894516 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.894538 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.896121 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.896126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.897790 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.897805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.898635 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.898643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.898977 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.899698 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:20:40.899728 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:40.902078 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:40.905218 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:40.905226 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:40.905242 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:40.905735 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:40.906085 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:20:40.906088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:20:40.906090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:20:40.906283 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:20:40.906759 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:40.906766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:40.907138 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 12:20:40.907390 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 12:20:40.907859 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":287,"threads_started":1}
I20260811 12:20:40.910804 7023 heartbeater.cc:347] Connected to a master server at 127.7.39.62:33407
I20260811 12:20:40.911136 7023 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:40.912113 7023 heartbeater.cc:511] Master 127.7.39.62:33407 requested a full tablet report, sending...
I20260811 12:20:40.913717 6256 ts_manager.cc:194] Registered new tserver with Master: 03d6d0e8820f44ffacf0c56d0462de41 (127.7.39.3:45645)
I20260811 12:20:40.914669 6256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.39.3@KRBTEST.COM'} at 127.7.39.3:56291
I20260811 12:20:40.924950 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:20:40.942612 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.929596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43344 (local address 127.7.39.62:33407)
0811 12:20:40.929813 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:40.929820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:40.930166 (+ 346us) server_negotiation.cc:407] Connection header received
0811 12:20:40.930292 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:40.930298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:40.930391 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:40.930515 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:40.932039 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.932837 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.934262 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.934677 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.934778 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.937322 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:40.937328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:40.937331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:40.937372 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:40.939955 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:40.940636 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:40.940639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:40.940642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:40.940772 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:40.941125 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:40.941129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:40.941131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:40.941533 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:40.941718 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:40.942150 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 12:20:40.942367 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:20:40.947269 6256 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:20:40.954922 6256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:20:40.985379 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.968280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:33141 (local address 127.0.0.1:56846)
0811 12:20:40.976191 (+ 7911us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:40.976210 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:40.976355 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:40.977228 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:40.977233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:40.977263 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:40.977650 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.977663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.979916 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.979921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.982168 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.982183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.984304 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.984309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.984467 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.984507 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:20:40.984655 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 12:20:40.984877 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7764,"mutex_wait_us":125,"thread_start_us":7620,"threads_started":1}
I20260811 12:20:40.985821 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.969910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:34661 (local address 127.0.0.1:59764)
0811 12:20:40.976565 (+ 6655us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:40.976582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:40.976685 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:40.977710 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:40.977713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:40.977737 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:40.978073 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.978082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.979582 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.979587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.982755 (+ 3168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.982770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.984919 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.984925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.985126 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.985185 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 12:20:40.985276 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:20:40.985424 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6513,"mutex_wait_us":155,"thread_start_us":6313,"threads_started":1}
I20260811 12:20:40.985898 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.968384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.7.39.3:45645)
0811 12:20:40.975874 (+ 7490us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:40.975885 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:40.975938 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:20:40.976055 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:40.976064 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:40.976344 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:40.977434 (+ 1090us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:40.977725 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.979404 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.983027 (+ 3623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.984418 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.984574 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.984798 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:40.985056 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 12:20:40.985257 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7288,"thread_start_us":7147,"threads_started":1}
I20260811 12:20:40.986171 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.968875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.7.39.1:33141)
0811 12:20:40.969590 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:40.969601 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:40.976518 (+ 6917us) server_negotiation.cc:407] Connection header received
0811 12:20:40.976642 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:40.976649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:40.976901 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:40.977098 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:40.977854 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.979803 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.982345 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.983789 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.983914 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.984177 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:40.985686 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0811 12:20:40.985859 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":372,"threads_started":1}
I20260811 12:20:40.986500 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.968191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45645 (local address 127.0.0.1:55116)
0811 12:20:40.975008 (+ 6817us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:40.975071 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:40.975303 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:40.976649 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:40.976658 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:40.976712 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:40.977197 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.977212 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.979570 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.979575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.982755 (+ 3180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:40.982770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.984650 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.984657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:40.984829 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.984864 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:20:40.984948 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 12:20:40.985091 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6479,"mutex_wait_us":96,"spinlock_wait_cycles":5504,"thread_start_us":6293,"threads_started":1}
I20260811 12:20:40.986603 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:40.969511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.7.39.2:34661)
0811 12:20:40.977227 (+ 7716us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:40.977236 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:40.977266 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:20:40.977360 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:40.977367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:40.977529 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:40.977684 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:40.978234 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.979414 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.982955 (+ 3541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:40.984418 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:40.984574 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:40.984776 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:40.986161 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0811 12:20:40.986331 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7510,"thread_start_us":7391,"threads_started":1}
I20260811 12:20:40.990201 6913 tablet_service.cc:1467] Processing CreateTablet for tablet 07d8b1e428764d889623ddfdf0524cca (DEFAULT_TABLE table=test-table [id=8f1d5775450a462eab3cdc86a480a6d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:40.990334 6461 tablet_service.cc:1467] Processing CreateTablet for tablet 07d8b1e428764d889623ddfdf0524cca (DEFAULT_TABLE table=test-table [id=8f1d5775450a462eab3cdc86a480a6d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:40.990659 6689 tablet_service.cc:1467] Processing CreateTablet for tablet 07d8b1e428764d889623ddfdf0524cca (DEFAULT_TABLE table=test-table [id=8f1d5775450a462eab3cdc86a480a6d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:40.991509 6913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d8b1e428764d889623ddfdf0524cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:40.991912 6689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d8b1e428764d889623ddfdf0524cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:40.991904 6461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d8b1e428764d889623ddfdf0524cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:41.003809 7048 tablet_bootstrap.cc:492] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Bootstrap starting.
I20260811 12:20:41.006886 7049 tablet_bootstrap.cc:492] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: Bootstrap starting.
I20260811 12:20:41.008322 7050 tablet_bootstrap.cc:492] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Bootstrap starting.
I20260811 12:20:41.009035 7048 tablet_bootstrap.cc:654] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:41.010516 7048 log.cc:824] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:41.010943 7049 tablet_bootstrap.cc:654] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:41.011838 7050 tablet_bootstrap.cc:654] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:41.012383 7049 log.cc:824] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:41.013096 7050 log.cc:824] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:41.016911 7049 tablet_bootstrap.cc:492] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: No bootstrap required, opened a new log
I20260811 12:20:41.017313 7049 ts_tablet_manager.cc:1397] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260811 12:20:41.019829 7050 tablet_bootstrap.cc:492] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: No bootstrap required, opened a new log
I20260811 12:20:41.020118 7050 ts_tablet_manager.cc:1397] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 12:20:41.020156 7048 tablet_bootstrap.cc:492] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: No bootstrap required, opened a new log
I20260811 12:20:41.020427 7048 ts_tablet_manager.cc:1397] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.000s
I20260811 12:20:41.028455 7050 raft_consensus.cc:348] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.028422 7049 raft_consensus.cc:348] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.028932 7049 raft_consensus.cc:374] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:41.028939 7050 raft_consensus.cc:374] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:41.029049 7050 raft_consensus.cc:729] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6177fc53b6418c94a79c16601a9db5, State: Initialized, Role: FOLLOWER
I20260811 12:20:41.029049 7049 raft_consensus.cc:729] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31dc1247f94b499e9044043c0f7c02f0, State: Initialized, Role: FOLLOWER
I20260811 12:20:41.029779 7050 consensus_queue.cc:260] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.029942 7049 consensus_queue.cc:260] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.031088 6803 heartbeater.cc:503] Master 127.7.39.62:33407 was elected leader, sending a full tablet report...
I20260811 12:20:41.031265 7048 raft_consensus.cc:348] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.031733 7050 ts_tablet_manager.cc:1428] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 12:20:41.031757 7048 raft_consensus.cc:374] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:41.031860 7048 raft_consensus.cc:729] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d6d0e8820f44ffacf0c56d0462de41, State: Initialized, Role: FOLLOWER
I20260811 12:20:41.032465 7048 consensus_queue.cc:260] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.036573 7023 heartbeater.cc:503] Master 127.7.39.62:33407 was elected leader, sending a full tablet report...
I20260811 12:20:41.037446 7048 ts_tablet_manager.cc:1428] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Time spent starting tablet: real 0.017s user 0.018s sys 0.001s
I20260811 12:20:41.039364 7049 ts_tablet_manager.cc:1428] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: Time spent starting tablet: real 0.022s user 0.019s sys 0.000s
W20260811 12:20:41.071005 6807 tablet.cc:2369] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:41.130492 7059 raft_consensus.cc:482] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:41.130836 7059 raft_consensus.cc:504] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.132687 7059 leader_election.cc:288] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03d6d0e8820f44ffacf0c56d0462de41 (127.7.39.3:45645), 9f6177fc53b6418c94a79c16601a9db5 (127.7.39.2:34661)
W20260811 12:20:41.138345 7026 tablet.cc:2369] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:20:41.141580 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:41.133193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:39651 (local address 127.7.39.2:34661)
0811 12:20:41.133405 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:41.133412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:41.134255 (+ 843us) server_negotiation.cc:407] Connection header received
0811 12:20:41.134425 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:41.134431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:41.134503 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:41.134615 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:41.135494 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.136614 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.139130 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.140353 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.140515 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:41.140623 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:41.141134 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 12:20:41.141318 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:20:41.141592 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:41.133243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:34169 (local address 127.7.39.3:45645)
0811 12:20:41.133413 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:41.133418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:41.134205 (+ 787us) server_negotiation.cc:407] Connection header received
0811 12:20:41.134414 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:41.134421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:41.134503 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:41.134625 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:41.135468 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.136652 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.139308 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.140561 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.140687 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:41.140763 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:41.141145 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 12:20:41.141318 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:20:41.141865 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:41.132972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:45645 (local address 127.7.39.1:34169)
0811 12:20:41.134042 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:41.134058 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:41.134249 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:41.134652 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:41.134657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:41.134707 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:41.135274 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:41.135291 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.136824 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.136829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:41.139097 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:41.139116 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.140808 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.140814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:41.140971 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:41.141001 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:20:41.141083 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 12:20:41.141267 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":128,"spinlock_wait_cycles":18432,"thread_start_us":652,"threads_started":1}
I20260811 12:20:41.141886 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:41.133123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:34661 (local address 127.7.39.1:39651)
0811 12:20:41.134036 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:41.134058 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:41.134296 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:41.134652 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:41.134657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:41.134709 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:41.135322 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:41.135332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.136797 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.136803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:41.138935 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:41.138954 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.140638 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.140644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:41.140858 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:41.140899 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:20:41.141030 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 12:20:41.141258 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":112,"spinlock_wait_cycles":61184,"thread_start_us":465,"threads_started":1}
I20260811 12:20:41.142928 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d8b1e428764d889623ddfdf0524cca" candidate_uuid: "31dc1247f94b499e9044043c0f7c02f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03d6d0e8820f44ffacf0c56d0462de41" is_pre_election: true
I20260811 12:20:41.142941 6722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d8b1e428764d889623ddfdf0524cca" candidate_uuid: "31dc1247f94b499e9044043c0f7c02f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f6177fc53b6418c94a79c16601a9db5" is_pre_election: true
I20260811 12:20:41.143445 6722 raft_consensus.cc:2454] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31dc1247f94b499e9044043c0f7c02f0 in term 0.
I20260811 12:20:41.143446 6944 raft_consensus.cc:2454] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31dc1247f94b499e9044043c0f7c02f0 in term 0.
I20260811 12:20:41.144164 6377 leader_election.cc:302] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31dc1247f94b499e9044043c0f7c02f0, 9f6177fc53b6418c94a79c16601a9db5; no voters:
I20260811 12:20:41.144599 7059 raft_consensus.cc:2781] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:20:41.144788 7059 raft_consensus.cc:482] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:20:41.144898 7059 raft_consensus.cc:3037] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:41.147336 7059 raft_consensus.cc:504] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.147974 7059 leader_election.cc:288] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [CANDIDATE]: Term 1 election: Requested vote from peers 03d6d0e8820f44ffacf0c56d0462de41 (127.7.39.3:45645), 9f6177fc53b6418c94a79c16601a9db5 (127.7.39.2:34661)
I20260811 12:20:41.148660 6722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d8b1e428764d889623ddfdf0524cca" candidate_uuid: "31dc1247f94b499e9044043c0f7c02f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f6177fc53b6418c94a79c16601a9db5"
I20260811 12:20:41.148700 6944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d8b1e428764d889623ddfdf0524cca" candidate_uuid: "31dc1247f94b499e9044043c0f7c02f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03d6d0e8820f44ffacf0c56d0462de41"
I20260811 12:20:41.148912 6722 raft_consensus.cc:3037] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:41.148911 6944 raft_consensus.cc:3037] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:41.152297 6722 raft_consensus.cc:2454] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31dc1247f94b499e9044043c0f7c02f0 in term 1.
I20260811 12:20:41.152297 6944 raft_consensus.cc:2454] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31dc1247f94b499e9044043c0f7c02f0 in term 1.
I20260811 12:20:41.152884 6383 leader_election.cc:302] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03d6d0e8820f44ffacf0c56d0462de41, 31dc1247f94b499e9044043c0f7c02f0, 9f6177fc53b6418c94a79c16601a9db5; no voters:
I20260811 12:20:41.153302 7059 raft_consensus.cc:2781] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:41.153792 7059 raft_consensus.cc:686] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 1 LEADER]: Becoming Leader. State: Replica: 31dc1247f94b499e9044043c0f7c02f0, State: Running, Role: LEADER
I20260811 12:20:41.154294 7059 consensus_queue.cc:237] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } }
I20260811 12:20:41.159622 6259 catalog_manager.cc:5495] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31dc1247f94b499e9044043c0f7c02f0 (127.7.39.1). New cstate: current_term: 1 leader_uuid: "31dc1247f94b499e9044043c0f7c02f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31dc1247f94b499e9044043c0f7c02f0" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 33141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 } health_report { overall_health: UNKNOWN } } }
I20260811 12:20:41.189765 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:41.183387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.7.39.1:33141)
0811 12:20:41.183602 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:41.183608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:41.183630 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:20:41.183832 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:41.183836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:41.183927 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:41.184068 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:20:41.184455 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.185222 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.186798 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.187255 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.187376 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:41.188360 (+ 984us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:20:41.189035 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:20:41.189240 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:41.189362 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 12:20:41.189469 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:20:41.199649 6722 raft_consensus.cc:1261] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 1 FOLLOWER]: Refusing update from remote peer 31dc1247f94b499e9044043c0f7c02f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:41.199644 6944 raft_consensus.cc:1261] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 1 FOLLOWER]: Refusing update from remote peer 31dc1247f94b499e9044043c0f7c02f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:41.201120 7065 consensus_queue.cc:1035] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03d6d0e8820f44ffacf0c56d0462de41" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 45645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:41.201644 7059 consensus_queue.cc:1035] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f6177fc53b6418c94a79c16601a9db5" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 34661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:41.230993 6256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:41.231283 6256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:41.235198 6256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac0ddc7f26a348629cecd4719dccb783: Sending DeleteTablet for 3 replicas of tablet 07d8b1e428764d889623ddfdf0524cca
I20260811 12:20:41.236407 6913 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d8b1e428764d889623ddfdf0524cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:41 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:55116
I20260811 12:20:41.236842 6689 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d8b1e428764d889623ddfdf0524cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:41 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:59764
I20260811 12:20:41.236869 6461 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d8b1e428764d889623ddfdf0524cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:41 UTC) from {username='kudu', principal='kudu/127.7.39.62@KRBTEST.COM'} at 127.0.0.1:56846
I20260811 12:20:41.238845 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 6352
I20260811 12:20:41.245769 7084 tablet_replica.cc:330] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: stopping tablet replica
I20260811 12:20:41.246218 7084 raft_consensus.cc:2229] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:41.246757 7084 raft_consensus.cc:2258] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:41.253749 7085 tablet_replica.cc:330] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: stopping tablet replica
I20260811 12:20:41.253943 7083 tablet_replica.cc:330] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: stopping tablet replica
I20260811 12:20:41.254316 7083 raft_consensus.cc:2229] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:41.254324 7085 raft_consensus.cc:2229] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:20:41.254904 7083 raft_consensus.cc:2258] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:41.255623 7085 raft_consensus.cc:2258] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:41.256870 7084 ts_tablet_manager.cc:1905] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:41.257270 7083 ts_tablet_manager.cc:1905] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:41.257673 7085 ts_tablet_manager.cc:1905] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:20:41 dist-test-slave-rmhg krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.1@KRBTEST.COM
I20260811 12:20:41.264564 7083 ts_tablet_manager.cc:1918] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:41.264781 7085 ts_tablet_manager.cc:1918] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:41.264861 7083 log.cc:1196] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/07d8b1e428764d889623ddfdf0524cca
I20260811 12:20:41.264977 7085 log.cc:1196] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/07d8b1e428764d889623ddfdf0524cca
I20260811 12:20:41.265486 7083 ts_tablet_manager.cc:1939] T 07d8b1e428764d889623ddfdf0524cca P 03d6d0e8820f44ffacf0c56d0462de41: Deleting consensus metadata
I20260811 12:20:41.265549 7085 ts_tablet_manager.cc:1939] T 07d8b1e428764d889623ddfdf0524cca P 31dc1247f94b499e9044043c0f7c02f0: Deleting consensus metadata
I20260811 12:20:41.267668 6236 catalog_manager.cc:4841] TS 31dc1247f94b499e9044043c0f7c02f0 (127.7.39.1:33141): tablet 07d8b1e428764d889623ddfdf0524cca (table test-table [id=8f1d5775450a462eab3cdc86a480a6d2]) successfully deleted
I20260811 12:20:41.267680 6237 catalog_manager.cc:4841] TS 03d6d0e8820f44ffacf0c56d0462de41 (127.7.39.3:45645): tablet 07d8b1e428764d889623ddfdf0524cca (table test-table [id=8f1d5775450a462eab3cdc86a480a6d2]) successfully deleted
I20260811 12:20:41.268860 7084 ts_tablet_manager.cc:1918] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:41.269045 7084 log.cc:1196] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/07d8b1e428764d889623ddfdf0524cca
I20260811 12:20:41.269420 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:41.240077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56860 (local address 127.7.39.1:33141)
0811 12:20:41.240284 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:41.240290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:41.246571 (+ 6281us) server_negotiation.cc:407] Connection header received
0811 12:20:41.246776 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:41.246781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:41.246902 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:41.247025 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:41.249319 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.250585 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.252596 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.253436 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.253558 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:41.260478 (+ 6920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:41.260495 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:41.260514 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:41.260573 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:41.264747 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:41.266285 (+ 1538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:41.266296 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:41.266300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:41.266501 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:41.267062 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:41.267067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:41.267070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:41.267585 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:41.268030 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:41.268707 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 12:20:41.269012 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:20:41.269706 7084 ts_tablet_manager.cc:1939] T 07d8b1e428764d889623ddfdf0524cca P 9f6177fc53b6418c94a79c16601a9db5: Deleting consensus metadata
I20260811 12:20:41.271152 6562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:41.271509 6234 catalog_manager.cc:4841] TS 9f6177fc53b6418c94a79c16601a9db5 (127.7.39.2:34661): tablet 07d8b1e428764d889623ddfdf0524cca (table test-table [id=8f1d5775450a462eab3cdc86a480a6d2]) successfully deleted
I20260811 12:20:41.370882 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 6352
I20260811 12:20:41.396649 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 6579
Aug 11 12:20:41 dist-test-slave-rmhg krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.2@KRBTEST.COM
I20260811 12:20:41.413340 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:41.397662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.7.39.2:34661)
0811 12:20:41.397833 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:41.397838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:41.397858 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:20:41.397923 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:41.397927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:41.398016 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:41.398127 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:41.399601 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.400429 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.401917 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.402338 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.402439 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:41.406933 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:41.406951 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:41.406972 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:41.407035 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:41.409983 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:41.410852 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:41.410861 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:41.410863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:41.411044 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:41.411511 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:41.411515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:41.411518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:41.412004 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:41.412299 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:41.412711 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 12:20:41.412912 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:20:41.414208 6796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:41.505308 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 6579
I20260811 12:20:41.529601 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 6815
Aug 11 12:20:41 dist-test-slave-rmhg krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.3@KRBTEST.COM
I20260811 12:20:41.545225 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:41.530543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.7.39.3:45645)
0811 12:20:41.530723 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:41.530728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:41.530747 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:20:41.530895 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:41.530899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:41.530968 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:41.531054 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:41.532663 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.533483 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.534895 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:41.535329 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:41.535433 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:41.539944 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:41.539962 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:41.539983 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:41.540023 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:41.542661 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:41.543342 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:41.543349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:41.543352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:41.543483 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:41.543819 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:41.543822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:41.543824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:41.544192 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:41.544435 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:41.544685 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 12:20:41.544862 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:20:41.545941 7014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:41.635672 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 6815
I20260811 12:20:41.657423 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 6205
I20260811 12:20:41.658509 6306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:41.770398 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 6205
2026-08-11T12:20:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4784 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T12:20:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:41Z Disabled control of system clock
I20260811 12:20:41.872243 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:41691
--webserver_interface=127.7.39.62
--webserver_port=0
--builtin_ntp_servers=127.7.39.20:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:41691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:20:42.234768 7105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:42.235097 7105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:42.235179 7105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:42.242827 7105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:42.242971 7105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:42.243009 7105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:42.243073 7105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:42.243119 7105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:42.252985 7105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:41691
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:41691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:42.254473 7105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:42.256363 7105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:42.268796 7110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:42.268829 7116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:42.268865 7111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:42.269855 7105 server_base.cc:1034] running on GCE node
I20260811 12:20:42.270784 7105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:42.272440 7105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:42.273660 7105 hybrid_clock.cc:648] HybridClock initialized: now 1786450842273599 us; error 65 us; skew 500 ppm
I20260811 12:20:42.274039 7105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:42.280470 7105 webserver.cc:460] Webserver started at http://127.7.39.62:37457/ using document root <none> and password file <none>
I20260811 12:20:42.281378 7105 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:42.281515 7105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:42.281883 7105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:42.285207 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "c137e31df96443e899101941bbdbe0d9"
format_stamp: "Formatted at 2026-08-11 12:20:42 on dist-test-slave-rmhg"
server_key: "a0fb46f7c93ef5331f2f9d804447e97c"
server_key_iv: "821a91b692ee5a2c49989456e135c65d"
server_key_version: "encryptionkey@0"
I20260811 12:20:42.286178 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "c137e31df96443e899101941bbdbe0d9"
format_stamp: "Formatted at 2026-08-11 12:20:42 on dist-test-slave-rmhg"
server_key: "a0fb46f7c93ef5331f2f9d804447e97c"
server_key_iv: "821a91b692ee5a2c49989456e135c65d"
server_key_version: "encryptionkey@0"
I20260811 12:20:42.292476 7105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 12:20:42.296800 7124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:42.298481 7105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:20:42.298708 7105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "c137e31df96443e899101941bbdbe0d9"
format_stamp: "Formatted at 2026-08-11 12:20:42 on dist-test-slave-rmhg"
server_key: "a0fb46f7c93ef5331f2f9d804447e97c"
server_key_iv: "821a91b692ee5a2c49989456e135c65d"
server_key_version: "encryptionkey@0"
I20260811 12:20:42.298952 7105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:42.323308 7105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:42.324307 7105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:42.324651 7105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:42.344521 7105 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:41691
I20260811 12:20:42.344542 7203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:41691 every 8 connection(s)
I20260811 12:20:42.346055 7105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:42.346448 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 7105
I20260811 12:20:42.346637 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:42.347247 7324 external_mini_cluster.cc:1428] Setting key 8ad16cdde314df193505b7aa6e6dc356
I20260811 12:20:42.350968 7204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:42.361748 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:42.349417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52668 (local address 127.7.39.62:41691)
0811 12:20:42.350588 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:42.350600 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:42.350646 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:20:42.352439 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:42.352480 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:42.353008 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:42.353490 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:42.354123 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:42.355546 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:42.356991 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:42.357750 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:42.357938 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:42.359175 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:42.359185 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:42.359200 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:42.359834 (+ 634us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:42.359923 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:42.359950 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:42.360062 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:42.360318 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 12:20:42.360672 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":438,"threads_started":1}
I20260811 12:20:42.362783 7204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9: Bootstrap starting.
I20260811 12:20:42.366210 7204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:42.367188 7204 log.cc:824] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:42.370245 7204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9: No bootstrap required, opened a new log
I20260811 12:20:42.378072 7204 raft_consensus.cc:348] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c137e31df96443e899101941bbdbe0d9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41691 } }
I20260811 12:20:42.378538 7204 raft_consensus.cc:374] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:42.378643 7204 raft_consensus.cc:729] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c137e31df96443e899101941bbdbe0d9, State: Initialized, Role: FOLLOWER
I20260811 12:20:42.379374 7204 consensus_queue.cc:260] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c137e31df96443e899101941bbdbe0d9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41691 } }
I20260811 12:20:42.379679 7204 raft_consensus.cc:388] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:42.379849 7204 raft_consensus.cc:482] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:42.380046 7204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:42.383383 7204 raft_consensus.cc:504] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c137e31df96443e899101941bbdbe0d9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41691 } }
I20260811 12:20:42.384100 7204 leader_election.cc:302] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c137e31df96443e899101941bbdbe0d9; no voters:
I20260811 12:20:42.384917 7204 leader_election.cc:288] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:42.385076 7210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:42.385815 7210 raft_consensus.cc:686] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [term 1 LEADER]: Becoming Leader. State: Replica: c137e31df96443e899101941bbdbe0d9, State: Running, Role: LEADER
I20260811 12:20:42.386374 7210 consensus_queue.cc:237] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c137e31df96443e899101941bbdbe0d9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41691 } }
I20260811 12:20:42.387176 7204 sys_catalog.cc:564] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:42.389290 7213 sys_catalog.cc:455] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c137e31df96443e899101941bbdbe0d9. Latest consensus state: current_term: 1 leader_uuid: "c137e31df96443e899101941bbdbe0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c137e31df96443e899101941bbdbe0d9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41691 } } }
I20260811 12:20:42.389765 7213 sys_catalog.cc:458] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:42.389878 7212 sys_catalog.cc:455] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c137e31df96443e899101941bbdbe0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c137e31df96443e899101941bbdbe0d9" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 41691 } } }
I20260811 12:20:42.390216 7212 sys_catalog.cc:458] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:42.392604 7222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:42.398450 7222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:42.405303 7241 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:20:42.407673 7222 catalog_manager.cc:1347] Generated new cluster ID: 1af07f37799e4572870bdbab488835f3
I20260811 12:20:42.407814 7222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:42.422008 7222 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:42.423559 7222 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:42.432276 7222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9: Generated new TSK 0
I20260811 12:20:42.433310 7222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:42.447165 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.1:0
--local_ip_for_outbound_sockets=127.7.39.1
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41691
--builtin_ntp_servers=127.7.39.20:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:20:42.813931 7244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:42.814250 7244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:42.814340 7244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:42.821915 7244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:42.822044 7244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:42.822166 7244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.1
I20260811 12:20:42.832005 7244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.39.1
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.1
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:42.833411 7244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:42.835351 7244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:42.849236 7254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:42.849282 7255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:42.849442 7257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:42.850137 7244 server_base.cc:1034] running on GCE node
I20260811 12:20:42.850831 7244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:42.851852 7244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:42.853076 7244 hybrid_clock.cc:648] HybridClock initialized: now 1786450842853018 us; error 52 us; skew 500 ppm
I20260811 12:20:42.853451 7244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:42.855878 7244 webserver.cc:460] Webserver started at http://127.7.39.1:45591/ using document root <none> and password file <none>
I20260811 12:20:42.856604 7244 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:42.856714 7244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:42.857038 7244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:42.860149 7244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/data/instance:
uuid: "517acd0d1a0341249e84d3f03811e53b"
format_stamp: "Formatted at 2026-08-11 12:20:42 on dist-test-slave-rmhg"
server_key: "8f9ae8da69ac6087ded8154308d2a653"
server_key_iv: "a016bb7d566f681f5e46010cc727a758"
server_key_version: "encryptionkey@0"
I20260811 12:20:42.860973 7244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance:
uuid: "517acd0d1a0341249e84d3f03811e53b"
format_stamp: "Formatted at 2026-08-11 12:20:42 on dist-test-slave-rmhg"
server_key: "8f9ae8da69ac6087ded8154308d2a653"
server_key_iv: "a016bb7d566f681f5e46010cc727a758"
server_key_version: "encryptionkey@0"
I20260811 12:20:42.866634 7244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:20:42.870440 7267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:42.871986 7244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:42.872167 7244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/wal
uuid: "517acd0d1a0341249e84d3f03811e53b"
format_stamp: "Formatted at 2026-08-11 12:20:42 on dist-test-slave-rmhg"
server_key: "8f9ae8da69ac6087ded8154308d2a653"
server_key_iv: "a016bb7d566f681f5e46010cc727a758"
server_key_version: "encryptionkey@0"
I20260811 12:20:42.872402 7244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:42.893287 7244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:42.894176 7244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:42.894466 7244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:42.897495 7244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:42.899210 7244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:42.899322 7244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:42.899432 7244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:42.899501 7244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:42.931676 7244 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.1:42959
I20260811 12:20:42.931744 7463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.1:42959 every 8 connection(s)
I20260811 12:20:42.933058 7244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/data/info.pb
I20260811 12:20:42.939090 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 7244
I20260811 12:20:42.939260 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/wal/instance
I20260811 12:20:42.939723 7324 external_mini_cluster.cc:1428] Setting key a5b0c2f043864aadf4f23f6922f88c79
I20260811 12:20:42.942631 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.2:0
--local_ip_for_outbound_sockets=127.7.39.2
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41691
--builtin_ntp_servers=127.7.39.20:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:20:42.946228 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:42.935703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:43979 (local address 127.7.39.62:41691)
0811 12:20:42.936144 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:42.936148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:42.937523 (+ 1375us) server_negotiation.cc:407] Connection header received
0811 12:20:42.939374 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:42.939380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:42.939464 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:42.939563 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:42.940804 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:42.941721 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:42.943399 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:42.943805 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:42.943916 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:42.945353 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:42.945359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:42.945362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:42.945382 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:42.945424 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:42.945437 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:42.945514 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:42.945749 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 12:20:42.945855 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":211,"threads_started":1}
I20260811 12:20:42.947328 7468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:42.936351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41691 (local address 127.7.39.1:43979)
0811 12:20:42.937360 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:42.937409 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:42.939053 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:42.939833 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:42.939845 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:42.939948 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:42.940617 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:42.940643 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:42.941879 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:42.941883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:42.943255 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:42.943269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:42.944035 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:42.944042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:42.944272 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:42.944978 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:42.944981 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:42.945026 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:42.945120 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:42.945528 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:42.945541 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:42.945550 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:20:42.945728 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 12:20:42.946173 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":282,"threads_started":1}
I20260811 12:20:42.948976 7464 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41691
I20260811 12:20:42.949354 7464 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:42.950156 7464 heartbeater.cc:511] Master 127.7.39.62:41691 requested a full tablet report, sending...
I20260811 12:20:42.952455 7154 ts_manager.cc:194] Registered new tserver with Master: 517acd0d1a0341249e84d3f03811e53b (127.7.39.1:42959)
I20260811 12:20:42.954039 7154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.1:43979
W20260811 12:20:43.300714 7469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:43.301055 7469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:43.301191 7469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:43.308935 7469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:43.309082 7469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:43.309257 7469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.2
I20260811 12:20:43.319254 7469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.39.2
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.2
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:43.320730 7469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:43.322824 7469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:43.337414 7475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:43.337451 7481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:43.337546 7474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:43.338217 7469 server_base.cc:1034] running on GCE node
I20260811 12:20:43.338879 7469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:43.339949 7469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:43.341259 7469 hybrid_clock.cc:648] HybridClock initialized: now 1786450843341193 us; error 47 us; skew 500 ppm
I20260811 12:20:43.341629 7469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:43.344322 7469 webserver.cc:460] Webserver started at http://127.7.39.2:44913/ using document root <none> and password file <none>
I20260811 12:20:43.345077 7469 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:43.345249 7469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:43.345598 7469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:43.348706 7469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/data/instance:
uuid: "c98177e220b4453197238e70457db360"
format_stamp: "Formatted at 2026-08-11 12:20:43 on dist-test-slave-rmhg"
server_key: "043b7c194ba33f6615fc5805c822ae4d"
server_key_iv: "c692594e197de91a879e49ac9a0c13e1"
server_key_version: "encryptionkey@0"
I20260811 12:20:43.349623 7469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance:
uuid: "c98177e220b4453197238e70457db360"
format_stamp: "Formatted at 2026-08-11 12:20:43 on dist-test-slave-rmhg"
server_key: "043b7c194ba33f6615fc5805c822ae4d"
server_key_iv: "c692594e197de91a879e49ac9a0c13e1"
server_key_version: "encryptionkey@0"
I20260811 12:20:43.355772 7469 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 12:20:43.359772 7492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:43.361346 7469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:20:43.361537 7469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/wal
uuid: "c98177e220b4453197238e70457db360"
format_stamp: "Formatted at 2026-08-11 12:20:43 on dist-test-slave-rmhg"
server_key: "043b7c194ba33f6615fc5805c822ae4d"
server_key_iv: "c692594e197de91a879e49ac9a0c13e1"
server_key_version: "encryptionkey@0"
I20260811 12:20:43.361774 7469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:43.375991 7469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:43.376971 7469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:43.377307 7469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:43.378379 7469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:43.379995 7469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:43.380087 7469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:43.380211 7469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:43.380280 7469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:43.412998 7469 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.2:36783
I20260811 12:20:43.413096 7680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.2:36783 every 8 connection(s)
I20260811 12:20:43.414415 7469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/data/info.pb
I20260811 12:20:43.417223 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 7469
I20260811 12:20:43.417419 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/wal/instance
I20260811 12:20:43.417892 7324 external_mini_cluster.cc:1428] Setting key 2e1156336189154c3fd6722fe2088467
I20260811 12:20:43.421286 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.39.3:0
--local_ip_for_outbound_sockets=127.7.39.3
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41691
--builtin_ntp_servers=127.7.39.20:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:20:43.429010 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.417064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.2:36457 (local address 127.7.39.62:41691)
0811 12:20:43.417327 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:43.417334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:43.418953 (+ 1619us) server_negotiation.cc:407] Connection header received
0811 12:20:43.421282 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:43.421286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:43.421362 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:43.421473 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:43.422977 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.423761 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.425966 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.426385 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.426483 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.428247 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:43.428253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:43.428256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:43.428279 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:43.428315 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:43.428327 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:43.428398 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:43.428668 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 12:20:43.428764 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 12:20:43.430584 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.417683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41691 (local address 127.7.39.2:36457)
0811 12:20:43.418767 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:43.418828 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:43.420902 (+ 2074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:43.421765 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:43.421779 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:43.421918 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:43.422783 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.422810 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.423934 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.423938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.425824 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.425840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.426620 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.426628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.426925 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.427805 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:43.427808 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:43.427866 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:43.427978 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:43.428419 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:43.428430 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:43.428439 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:20:43.428641 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 12:20:43.429098 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":344,"threads_started":1}
I20260811 12:20:43.432363 7684 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41691
I20260811 12:20:43.432721 7684 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:43.433738 7684 heartbeater.cc:511] Master 127.7.39.62:41691 requested a full tablet report, sending...
I20260811 12:20:43.435392 7154 ts_manager.cc:194] Registered new tserver with Master: c98177e220b4453197238e70457db360 (127.7.39.2:36783)
I20260811 12:20:43.436488 7154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.2:36457
W20260811 12:20:43.779001 7688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:43.779327 7688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:43.779417 7688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:43.786972 7688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:43.787111 7688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:43.787269 7688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.39.3
I20260811 12:20:43.797137 7688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.39.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.39.3
--webserver_port=0
--tserver_master_addrs=127.7.39.62:41691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.39.3
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:43.798594 7688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:43.800508 7688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:43.814270 7700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:43.814194 7706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:43.814219 7702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:43.815971 7688 server_base.cc:1034] running on GCE node
I20260811 12:20:43.816671 7688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:43.817806 7688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:43.819077 7688 hybrid_clock.cc:648] HybridClock initialized: now 1786450843819022 us; error 46 us; skew 500 ppm
I20260811 12:20:43.819479 7688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:20:43.822253 7688 webserver.cc:460] Webserver started at http://127.7.39.3:35785/ using document root <none> and password file <none>
I20260811 12:20:43.823071 7688 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:43.823212 7688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:43.823568 7688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:43.826895 7688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/data/instance:
uuid: "0395963128044d35960f3cadaeb8fc02"
format_stamp: "Formatted at 2026-08-11 12:20:43 on dist-test-slave-rmhg"
server_key: "2fbe286982ff4d4bc9c185fbae68f79e"
server_key_iv: "a61582fcd80fb32ce788331216fcb0bf"
server_key_version: "encryptionkey@0"
I20260811 12:20:43.827814 7688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance:
uuid: "0395963128044d35960f3cadaeb8fc02"
format_stamp: "Formatted at 2026-08-11 12:20:43 on dist-test-slave-rmhg"
server_key: "2fbe286982ff4d4bc9c185fbae68f79e"
server_key_iv: "a61582fcd80fb32ce788331216fcb0bf"
server_key_version: "encryptionkey@0"
I20260811 12:20:43.833700 7688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:20:43.837498 7717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:43.838954 7688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:43.839155 7688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/wal
uuid: "0395963128044d35960f3cadaeb8fc02"
format_stamp: "Formatted at 2026-08-11 12:20:43 on dist-test-slave-rmhg"
server_key: "2fbe286982ff4d4bc9c185fbae68f79e"
server_key_iv: "a61582fcd80fb32ce788331216fcb0bf"
server_key_version: "encryptionkey@0"
I20260811 12:20:43.839398 7688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:43.880324 7688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:43.881309 7688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:43.881603 7688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:43.882652 7688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:20:43.884261 7688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:20:43.884362 7688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:43.884500 7688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:20:43.884577 7688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:43.917176 7688 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.3:44055
I20260811 12:20:43.917285 7902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.3:44055 every 8 connection(s)
I20260811 12:20:43.918560 7688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/data/info.pb
I20260811 12:20:43.927830 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 7688
I20260811 12:20:43.928036 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/wal/instance
I20260811 12:20:43.928610 7324 external_mini_cluster.cc:1428] Setting key 05940243a8d56761e3ebafd18442ddb4
I20260811 12:20:43.931871 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.921327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.3:48821 (local address 127.7.39.62:41691)
0811 12:20:43.921571 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:43.921578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:43.923052 (+ 1474us) server_negotiation.cc:407] Connection header received
0811 12:20:43.924808 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:43.924812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:43.924889 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:43.924979 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:43.926035 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.926764 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.928813 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.929329 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.929436 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.930968 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:43.930974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:43.930977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:43.930995 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:43.931039 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:43.931056 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:43.931138 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:43.931454 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 12:20:43.931575 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:20:43.933573 7907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.921945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:41691 (local address 127.7.39.3:48821)
0811 12:20:43.922892 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:43.922929 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:43.924566 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:43.925188 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:43.925200 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:43.925298 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:20:43.925894 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.925911 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.926903 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.926907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.928673 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.928688 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.929561 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.929568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.929811 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.930620 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:20:43.930623 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:20:43.930680 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:20:43.930779 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:20:43.931195 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:20:43.931205 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:20:43.931213 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:20:43.932140 (+ 927us) client_negotiation.cc:240] Negotiation successful
0811 12:20:43.932486 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":278,"threads_started":1}
I20260811 12:20:43.934958 7903 heartbeater.cc:347] Connected to a master server at 127.7.39.62:41691
I20260811 12:20:43.935251 7903 heartbeater.cc:464] Registering TS with master...
I20260811 12:20:43.936034 7903 heartbeater.cc:511] Master 127.7.39.62:41691 requested a full tablet report, sending...
I20260811 12:20:43.937394 7154 ts_manager.cc:194] Registered new tserver with Master: 0395963128044d35960f3cadaeb8fc02 (127.7.39.3:44055)
I20260811 12:20:43.938199 7154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.39.3:48821
I20260811 12:20:43.944195 7324 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:20:43.954460 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.948546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.7.39.62:41691)
0811 12:20:43.948746 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:43.948750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:43.949216 (+ 466us) server_negotiation.cc:407] Connection header received
0811 12:20:43.949364 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:43.949369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:43.949440 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:43.949538 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:43.950103 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.950820 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.952222 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.952651 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.952752 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.953861 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:43.953866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:43.953869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:43.953889 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:43.953923 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:43.953936 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:43.954008 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:43.954120 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 12:20:43.954208 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:20:43.957327 7464 heartbeater.cc:503] Master 127.7.39.62:41691 was elected leader, sending a full tablet report...
I20260811 12:20:43.958510 7154 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:20:43.965430 7154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:20:43.988476 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.977411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.1:42959 (local address 127.0.0.1:60712)
0811 12:20:43.978200 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:43.978258 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:43.978438 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:43.979831 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:43.979838 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:43.979870 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:43.980246 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.980259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.982576 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.982582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.985368 (+ 2786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.985387 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.987319 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.987325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.987606 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.987681 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 12:20:43.987839 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 12:20:43.988067 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":60,"thread_start_us":318,"threads_started":1}
I20260811 12:20:43.989288 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.977535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60712 (local address 127.7.39.1:42959)
0811 12:20:43.978199 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:43.978208 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:43.978361 (+ 153us) server_negotiation.cc:407] Connection header received
0811 12:20:43.979315 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:43.979324 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:43.979579 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:43.980467 (+ 888us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:43.980728 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.982413 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.985642 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.987065 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.988276 (+ 1211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.988621 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:43.988714 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 12:20:43.988917 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":297,"threads_started":1}
I20260811 12:20:43.990913 7923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.978483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36783 (local address 127.0.0.1:42888)
0811 12:20:43.979027 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:43.979048 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:43.979161 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:43.980066 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:43.980070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:43.980099 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:43.980515 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.980529 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.982576 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.982582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.985519 (+ 2937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.985532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.989120 (+ 3588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.989127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.989332 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.989372 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:20:43.990392 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0811 12:20:43.990543 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":753,"thread_start_us":249,"threads_started":1}
I20260811 12:20:43.991003 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.978745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.7.39.2:36783)
0811 12:20:43.979230 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:43.979238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:43.979265 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:20:43.979356 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:43.979362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:43.979572 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:43.979788 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:43.980720 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.982401 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.985742 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.988597 (+ 2855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.988740 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.988979 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:43.990485 (+ 1506us) server_negotiation.cc:299] Negotiation successful
0811 12:20:43.990671 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":252,"threads_started":1}
I20260811 12:20:43.992699 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.979052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:44055 (local address 127.0.0.1:36190)
0811 12:20:43.983080 (+ 4028us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:43.983105 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:43.983229 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:43.984343 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:43.984347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:43.984380 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:43.984807 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.984818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.986610 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.986615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.989443 (+ 2828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:43.989457 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.991907 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.991914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:43.992095 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.992128 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:20:43.992221 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:20:43.992349 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3884,"mutex_wait_us":3,"thread_start_us":3730,"threads_started":1}
I20260811 12:20:43.992848 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:43.979655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.7.39.3:44055)
0811 12:20:43.983692 (+ 4037us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:43.983700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:43.983737 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:20:43.983843 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:43.983851 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:43.984143 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:43.984347 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:43.985007 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.986404 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.989686 (+ 3282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:43.991350 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:43.991492 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:43.991766 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:43.992286 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 12:20:43.992459 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3847,"thread_start_us":4361,"threads_started":1}
I20260811 12:20:43.993813 7349 tablet_service.cc:1467] Processing CreateTablet for tablet 653d83a5a43743d1bd45d79265ba1b05 (DEFAULT_TABLE table=test-table [id=d152ead36ea1448f9cbe1a9e899a0fba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:43.995507 7349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 653d83a5a43743d1bd45d79265ba1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:43.995440 7577 tablet_service.cc:1467] Processing CreateTablet for tablet 653d83a5a43743d1bd45d79265ba1b05 (DEFAULT_TABLE table=test-table [id=d152ead36ea1448f9cbe1a9e899a0fba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:43.996901 7577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 653d83a5a43743d1bd45d79265ba1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:43.996747 7802 tablet_service.cc:1467] Processing CreateTablet for tablet 653d83a5a43743d1bd45d79265ba1b05 (DEFAULT_TABLE table=test-table [id=d152ead36ea1448f9cbe1a9e899a0fba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:20:43.998286 7802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 653d83a5a43743d1bd45d79265ba1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:44.010319 7929 tablet_bootstrap.cc:492] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: Bootstrap starting.
I20260811 12:20:44.012368 7931 tablet_bootstrap.cc:492] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: Bootstrap starting.
I20260811 12:20:44.013940 7932 tablet_bootstrap.cc:492] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: Bootstrap starting.
I20260811 12:20:44.015978 7929 tablet_bootstrap.cc:654] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:44.016309 7931 tablet_bootstrap.cc:654] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:44.017277 7932 tablet_bootstrap.cc:654] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:44.017496 7929 log.cc:824] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:44.017511 7931 log.cc:824] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:44.018337 7932 log.cc:824] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:44.020913 7931 tablet_bootstrap.cc:492] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: No bootstrap required, opened a new log
I20260811 12:20:44.020916 7929 tablet_bootstrap.cc:492] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: No bootstrap required, opened a new log
I20260811 12:20:44.021238 7931 ts_tablet_manager.cc:1397] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260811 12:20:44.021257 7929 ts_tablet_manager.cc:1397] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 12:20:44.022419 7932 tablet_bootstrap.cc:492] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: No bootstrap required, opened a new log
I20260811 12:20:44.022668 7932 ts_tablet_manager.cc:1397] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260811 12:20:44.030274 7932 raft_consensus.cc:348] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.030704 7932 raft_consensus.cc:374] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:44.030804 7932 raft_consensus.cc:729] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0395963128044d35960f3cadaeb8fc02, State: Initialized, Role: FOLLOWER
I20260811 12:20:44.031476 7932 consensus_queue.cc:260] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.032137 7929 raft_consensus.cc:348] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.032630 7929 raft_consensus.cc:374] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:44.032734 7903 heartbeater.cc:503] Master 127.7.39.62:41691 was elected leader, sending a full tablet report...
I20260811 12:20:44.032748 7929 raft_consensus.cc:729] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c98177e220b4453197238e70457db360, State: Initialized, Role: FOLLOWER
I20260811 12:20:44.032141 7931 raft_consensus.cc:348] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.033437 7932 ts_tablet_manager.cc:1428] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: Time spent starting tablet: real 0.011s user 0.009s sys 0.003s
I20260811 12:20:44.033547 7931 raft_consensus.cc:374] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:44.033444 7929 consensus_queue.cc:260] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.033761 7931 raft_consensus.cc:729] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517acd0d1a0341249e84d3f03811e53b, State: Initialized, Role: FOLLOWER
I20260811 12:20:44.034329 7931 consensus_queue.cc:260] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.035012 7684 heartbeater.cc:503] Master 127.7.39.62:41691 was elected leader, sending a full tablet report...
I20260811 12:20:44.037624 7931 ts_tablet_manager.cc:1428] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
I20260811 12:20:44.038894 7929 ts_tablet_manager.cc:1428] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: Time spent starting tablet: real 0.018s user 0.014s sys 0.000s
I20260811 12:20:44.053117 7943 raft_consensus.cc:482] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:20:44.053471 7943 raft_consensus.cc:504] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.055154 7943 leader_election.cc:288] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c98177e220b4453197238e70457db360 (127.7.39.2:36783), 0395963128044d35960f3cadaeb8fc02 (127.7.39.3:44055)
I20260811 12:20:44.061741 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:44.055385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.2:36783 (local address 127.7.39.1:35681)
0811 12:20:44.055935 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:44.055951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:44.056116 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:44.056589 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:44.056594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:44.056662 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:44.057052 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:44.057067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.058209 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.058215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:44.059795 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:44.059809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.060990 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.060997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:44.061208 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:44.061244 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:20:44.061335 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:20:44.061456 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":279,"threads_started":1}
I20260811 12:20:44.061803 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:44.055523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:35681 (local address 127.7.39.2:36783)
0811 12:20:44.055765 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:44.055772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:44.056050 (+ 278us) server_negotiation.cc:407] Connection header received
0811 12:20:44.056273 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:44.056279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:44.056374 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:44.056462 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:44.057222 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.058077 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.059990 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.060805 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.060899 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:44.060965 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:44.061382 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 12:20:44.061509 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:20:44.062013 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:44.056378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.39.1:39047 (local address 127.7.39.3:44055)
0811 12:20:44.056572 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:44.056578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:44.056751 (+ 173us) server_negotiation.cc:407] Connection header received
0811 12:20:44.056910 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:44.056916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:44.056992 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:44.057096 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:20:44.057656 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.058466 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.060238 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.061073 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.061261 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:44.061356 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:44.061651 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:20:44.061781 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:20:44.062057 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:44.056042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.3:44055 (local address 127.7.39.1:39047)
0811 12:20:44.056660 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 12:20:44.056674 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:20:44.056789 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:20:44.057125 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:20:44.057129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:20:44.057191 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:20:44.057518 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:44.057529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.058608 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.058612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:44.060038 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:20:44.060053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.061300 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.061305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:20:44.061487 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:44.061519 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:20:44.061610 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:20:44.061739 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":314,"threads_started":1}
I20260811 12:20:44.062768 7619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "653d83a5a43743d1bd45d79265ba1b05" candidate_uuid: "517acd0d1a0341249e84d3f03811e53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c98177e220b4453197238e70457db360" is_pre_election: true
I20260811 12:20:44.062987 7836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "653d83a5a43743d1bd45d79265ba1b05" candidate_uuid: "517acd0d1a0341249e84d3f03811e53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0395963128044d35960f3cadaeb8fc02" is_pre_election: true
I20260811 12:20:44.063265 7619 raft_consensus.cc:2454] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 517acd0d1a0341249e84d3f03811e53b in term 0.
I20260811 12:20:44.063493 7836 raft_consensus.cc:2454] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 517acd0d1a0341249e84d3f03811e53b in term 0.
I20260811 12:20:44.063957 7277 leader_election.cc:302] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 517acd0d1a0341249e84d3f03811e53b, c98177e220b4453197238e70457db360; no voters:
I20260811 12:20:44.064363 7943 raft_consensus.cc:2781] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:20:44.064554 7943 raft_consensus.cc:482] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:20:44.064705 7943 raft_consensus.cc:3037] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:44.067240 7943 raft_consensus.cc:504] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.067836 7943 leader_election.cc:288] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [CANDIDATE]: Term 1 election: Requested vote from peers c98177e220b4453197238e70457db360 (127.7.39.2:36783), 0395963128044d35960f3cadaeb8fc02 (127.7.39.3:44055)
I20260811 12:20:44.068344 7619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "653d83a5a43743d1bd45d79265ba1b05" candidate_uuid: "517acd0d1a0341249e84d3f03811e53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c98177e220b4453197238e70457db360"
I20260811 12:20:44.068423 7836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "653d83a5a43743d1bd45d79265ba1b05" candidate_uuid: "517acd0d1a0341249e84d3f03811e53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0395963128044d35960f3cadaeb8fc02"
I20260811 12:20:44.068543 7619 raft_consensus.cc:3037] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:44.068645 7836 raft_consensus.cc:3037] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:44.070956 7619 raft_consensus.cc:2454] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 517acd0d1a0341249e84d3f03811e53b in term 1.
I20260811 12:20:44.070988 7836 raft_consensus.cc:2454] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 517acd0d1a0341249e84d3f03811e53b in term 1.
I20260811 12:20:44.071430 7277 leader_election.cc:302] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 517acd0d1a0341249e84d3f03811e53b, c98177e220b4453197238e70457db360; no voters:
I20260811 12:20:44.071750 7943 raft_consensus.cc:2781] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:44.072220 7943 raft_consensus.cc:686] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 1 LEADER]: Becoming Leader. State: Replica: 517acd0d1a0341249e84d3f03811e53b, State: Running, Role: LEADER
I20260811 12:20:44.072778 7943 consensus_queue.cc:237] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } }
I20260811 12:20:44.077670 7152 catalog_manager.cc:5495] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b reported cstate change: term changed from 0 to 1, leader changed from <none> to 517acd0d1a0341249e84d3f03811e53b (127.7.39.1). New cstate: current_term: 1 leader_uuid: "517acd0d1a0341249e84d3f03811e53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "517acd0d1a0341249e84d3f03811e53b" member_type: VOTER last_known_addr { host: "127.7.39.1" port: 42959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 } health_report { overall_health: UNKNOWN } } }
I20260811 12:20:44.097110 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:44.091719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60716 (local address 127.7.39.1:42959)
0811 12:20:44.091932 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:44.091939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:44.091959 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:20:44.092060 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:44.092063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:44.092177 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:44.092311 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:20:44.092680 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.093439 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.094922 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.095377 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.095493 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:44.095986 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:20:44.096531 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:20:44.096685 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:44.096779 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 12:20:44.096862 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:20:44.105674 7836 raft_consensus.cc:1261] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 1 FOLLOWER]: Refusing update from remote peer 517acd0d1a0341249e84d3f03811e53b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:44.105674 7619 raft_consensus.cc:1261] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 1 FOLLOWER]: Refusing update from remote peer 517acd0d1a0341249e84d3f03811e53b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:20:44.106681 7946 consensus_queue.cc:1035] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c98177e220b4453197238e70457db360" member_type: VOTER last_known_addr { host: "127.7.39.2" port: 36783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:44.107118 7943 consensus_queue.cc:1035] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0395963128044d35960f3cadaeb8fc02" member_type: VOTER last_known_addr { host: "127.7.39.3" port: 44055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:20:44.133955 7154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:44.134302 7154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:20:44.138336 7154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c137e31df96443e899101941bbdbe0d9: Sending DeleteTablet for 3 replicas of tablet 653d83a5a43743d1bd45d79265ba1b05
I20260811 12:20:44.139621 7349 tablet_service.cc:1514] Processing DeleteTablet for tablet 653d83a5a43743d1bd45d79265ba1b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:44 UTC) from {username='slave'} at 127.0.0.1:60712
I20260811 12:20:44.140116 7802 tablet_service.cc:1514] Processing DeleteTablet for tablet 653d83a5a43743d1bd45d79265ba1b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:44 UTC) from {username='slave'} at 127.0.0.1:36190
I20260811 12:20:44.140865 7577 tablet_service.cc:1514] Processing DeleteTablet for tablet 653d83a5a43743d1bd45d79265ba1b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:20:44 UTC) from {username='slave'} at 127.0.0.1:42888
I20260811 12:20:44.140944 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 7244
I20260811 12:20:44.144847 7977 tablet_replica.cc:330] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: stopping tablet replica
I20260811 12:20:44.145437 7977 raft_consensus.cc:2229] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:44.146062 7977 raft_consensus.cc:2258] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:44.152092 7968 tablet_replica.cc:330] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: stopping tablet replica
I20260811 12:20:44.152531 7975 tablet_replica.cc:330] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: stopping tablet replica
I20260811 12:20:44.152654 7968 raft_consensus.cc:2229] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:20:44.153115 7975 raft_consensus.cc:2229] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:20:44.153441 7968 raft_consensus.cc:2258] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:44.153824 7975 raft_consensus.cc:2258] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:20:44.154062 7977 ts_tablet_manager.cc:1905] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:44.156220 7975 ts_tablet_manager.cc:1905] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:44.156988 7968 ts_tablet_manager.cc:1905] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:20:44.161296 7977 ts_tablet_manager.cc:1918] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:44.161487 7977 log.cc:1196] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-1/wal/wals/653d83a5a43743d1bd45d79265ba1b05
I20260811 12:20:44.162101 7977 ts_tablet_manager.cc:1939] T 653d83a5a43743d1bd45d79265ba1b05 P c98177e220b4453197238e70457db360: Deleting consensus metadata
I20260811 12:20:44.163125 7975 ts_tablet_manager.cc:1918] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:44.163302 7975 log.cc:1196] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-2/wal/wals/653d83a5a43743d1bd45d79265ba1b05
I20260811 12:20:44.163841 7975 ts_tablet_manager.cc:1939] T 653d83a5a43743d1bd45d79265ba1b05 P 0395963128044d35960f3cadaeb8fc02: Deleting consensus metadata
I20260811 12:20:44.164279 7131 catalog_manager.cc:4841] TS c98177e220b4453197238e70457db360 (127.7.39.2:36783): tablet 653d83a5a43743d1bd45d79265ba1b05 (table test-table [id=d152ead36ea1448f9cbe1a9e899a0fba]) successfully deleted
I20260811 12:20:44.165529 7968 ts_tablet_manager.cc:1918] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:20:44.165853 7968 log.cc:1196] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: Deleting WAL directory at /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450508543501-7324-0/minicluster-data/ts-0/wal/wals/653d83a5a43743d1bd45d79265ba1b05
I20260811 12:20:44.165815 7131 catalog_manager.cc:4841] TS 0395963128044d35960f3cadaeb8fc02 (127.7.39.3:44055): tablet 653d83a5a43743d1bd45d79265ba1b05 (table test-table [id=d152ead36ea1448f9cbe1a9e899a0fba]) successfully deleted
I20260811 12:20:44.166700 7968 ts_tablet_manager.cc:1939] T 653d83a5a43743d1bd45d79265ba1b05 P 517acd0d1a0341249e84d3f03811e53b: Deleting consensus metadata
I20260811 12:20:44.167333 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:44.142704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.7.39.1:42959)
0811 12:20:44.143280 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:44.143287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:44.144205 (+ 918us) server_negotiation.cc:407] Connection header received
0811 12:20:44.146794 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:44.146801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:44.146944 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:44.147046 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:44.147873 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.149278 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.153868 (+ 4590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.154570 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.154797 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:44.165421 (+ 10624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:44.165436 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:44.165455 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:44.166404 (+ 949us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:44.166463 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:44.166486 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:44.166602 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:44.166825 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 12:20:44.166948 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440}
I20260811 12:20:44.168071 7457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:44.168689 7131 catalog_manager.cc:4841] TS 517acd0d1a0341249e84d3f03811e53b (127.7.39.1:42959): tablet 653d83a5a43743d1bd45d79265ba1b05 (table test-table [id=d152ead36ea1448f9cbe1a9e899a0fba]) successfully deleted
I20260811 12:20:44.266160 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 7244
I20260811 12:20:44.293985 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 7469
I20260811 12:20:44.301048 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:44.294825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.7.39.2:36783)
0811 12:20:44.295025 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:44.295030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:44.295104 (+ 74us) server_negotiation.cc:407] Connection header received
0811 12:20:44.295260 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:44.295264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:44.295367 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:44.295467 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:44.295856 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.296657 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.298045 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.298470 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.298599 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:44.299697 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:44.299705 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:44.299722 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:44.300331 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:44.300381 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:44.300407 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:44.300493 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:44.300616 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 12:20:44.300714 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:20:44.301859 7674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:44.392465 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 7469
I20260811 12:20:44.421871 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 7688
I20260811 12:20:44.429385 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:44.422781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.7.39.3:44055)
0811 12:20:44.422976 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:44.422983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:44.423004 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:20:44.423201 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:44.423206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:44.423301 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:44.423391 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:44.423846 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.424652 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.426091 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:44.426529 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:44.426665 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:44.427854 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:44.427865 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:44.427882 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:44.428531 (+ 649us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:44.428596 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:44.428620 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:44.428732 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:44.428884 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 12:20:44.429022 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:20:44.430145 7897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:44.520671 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 7688
I20260811 12:20:44.551144 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 7105
I20260811 12:20:44.552342 7199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:44.664014 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 7105
2026-08-11T12:20:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2898 ms)
[----------] 6 tests from EncryptionPolicyTest (23368 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:44 dist-test-slave-rmhg krb5kdc[7991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:44 dist-test-slave-rmhg krb5kdc[7991](info): set up 2 sockets
Aug 11 12:20:44 dist-test-slave-rmhg krb5kdc[7991](info): commencing operation
krb5kdc: starting...
W20260811 12:20:45.942844 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:46 dist-test-slave-rmhg krb5kdc[7991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:20:46.285092 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:46.663915 8013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:46.664248 8013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:46.664397 8013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:46.672536 8013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:46.672694 8013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:46.672755 8013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:46.672847 8013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:46.672904 8013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:46.684185 8013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:46.685720 8013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:46.687135 8013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:46.700069 8020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:46.700133 8026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:46.700145 8022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:46.701447 8013 server_base.cc:1034] running on GCE node
I20260811 12:20:46.702389 8013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:46.704010 8013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:46.705267 8013 hybrid_clock.cc:648] HybridClock initialized: now 1786450846705221 us; error 40 us; skew 500 ppm
I20260811 12:20:46.705668 8013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:46 dist-test-slave-rmhg krb5kdc[7991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:46.709331 8013 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:20:46.711077 8013 webserver.cc:460] Webserver started at http://127.0.0.1:45527/ using document root <none> and password file <none>
I20260811 12:20:46.711836 8013 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:46.711953 8013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:46.712237 8013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:46.715556 8013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "c8f276d7302040c6bed7867fb022ac96"
format_stamp: "Formatted at 2026-08-11 12:20:46 on dist-test-slave-rmhg"
server_key: "c9be13ec10076c9e9393134e23b26a52"
server_key_iv: "7101bb895249b3448c40e79670ec2b20"
server_key_version: "encryptionkey@0"
I20260811 12:20:46.716362 8013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "c8f276d7302040c6bed7867fb022ac96"
format_stamp: "Formatted at 2026-08-11 12:20:46 on dist-test-slave-rmhg"
server_key: "c9be13ec10076c9e9393134e23b26a52"
server_key_iv: "7101bb895249b3448c40e79670ec2b20"
server_key_version: "encryptionkey@0"
I20260811 12:20:46.722080 8013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:20:46.725857 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:46.727312 8013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:20:46.727496 8013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "c8f276d7302040c6bed7867fb022ac96"
format_stamp: "Formatted at 2026-08-11 12:20:46 on dist-test-slave-rmhg"
server_key: "c9be13ec10076c9e9393134e23b26a52"
server_key_iv: "7101bb895249b3448c40e79670ec2b20"
server_key_version: "encryptionkey@0"
I20260811 12:20:46.727692 8013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:46.747578 8013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:46.752247 8013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:46.752528 8013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:46.771492 8013 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33319
I20260811 12:20:46.771528 8110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33319 every 8 connection(s)
I20260811 12:20:46.773097 8013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:46.777300 8114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:46.780892 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8013
I20260811 12:20:46.781144 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:46.782027 7324 external_mini_cluster.cc:1428] Setting key e39439c63a2d46b4b9b9396409984078
I20260811 12:20:46.791664 8114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96: Bootstrap starting.
I20260811 12:20:46.796144 8114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:46.797696 8114 log.cc:824] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:20:46 dist-test-slave-rmhg krb5kdc[7991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:20:46.801342 8114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96: No bootstrap required, opened a new log
I20260811 12:20:46.808140 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:46.784267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35819 (local address 127.0.0.1:33319)
0811 12:20:46.785133 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:46.785170 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:46.785228 (+ 58us) server_negotiation.cc:407] Connection header received
0811 12:20:46.786962 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:46.786993 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:46.787519 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:46.787934 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:46.789945 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:46.791414 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:46.793544 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:46.794353 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:46.794507 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:46.799808 (+ 5301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:46.799817 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:46.799836 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:46.799881 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:46.803100 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:46.804209 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:46.804217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:46.804224 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:46.804455 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:46.804984 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:46.804992 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:46.804994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:46.805707 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:46.805978 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:46.806505 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 12:20:46.806903 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":402,"threads_started":1}
I20260811 12:20:46.810920 8114 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f276d7302040c6bed7867fb022ac96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } }
I20260811 12:20:46.811403 8114 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:46.811496 8114 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8f276d7302040c6bed7867fb022ac96, State: Initialized, Role: FOLLOWER
I20260811 12:20:46.812124 8114 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f276d7302040c6bed7867fb022ac96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } }
I20260811 12:20:46.812374 8114 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:46.812537 8114 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:46.812723 8114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:46.812773 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:20:46.815037 8114 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f276d7302040c6bed7867fb022ac96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } }
I20260811 12:20:46.815542 8114 leader_election.cc:302] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8f276d7302040c6bed7867fb022ac96; no voters:
I20260811 12:20:46.816180 8114 leader_election.cc:288] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:46.816308 8127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:46.817515 8127 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [term 1 LEADER]: Becoming Leader. State: Replica: c8f276d7302040c6bed7867fb022ac96, State: Running, Role: LEADER
I20260811 12:20:46.818219 8127 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f276d7302040c6bed7867fb022ac96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } }
I20260811 12:20:46.819056 8114 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:46.822533 8134 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8f276d7302040c6bed7867fb022ac96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f276d7302040c6bed7867fb022ac96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } } }
I20260811 12:20:46.822971 8134 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:46.823496 8136 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8f276d7302040c6bed7867fb022ac96. Latest consensus state: current_term: 1 leader_uuid: "c8f276d7302040c6bed7867fb022ac96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f276d7302040c6bed7867fb022ac96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } } }
I20260811 12:20:46.823844 8136 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:46.833901 8143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:46.838866 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:46.818486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59063 (local address 127.0.0.1:33319)
0811 12:20:46.818654 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:46.818659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:46.818754 (+ 95us) server_negotiation.cc:407] Connection header received
0811 12:20:46.818938 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:46.818944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:46.819023 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:46.819139 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:46.821454 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:46.822917 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:46.825103 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:46.825911 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:46.826026 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:46.830180 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:46.830189 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:46.830195 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:46.830255 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:46.834377 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:46.835662 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:46.835669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:46.835673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:46.835876 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:46.836510 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:46.836516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:46.836520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:46.837068 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:46.837422 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:46.838192 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 12:20:46.838409 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:20:46.841168 8143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:46.851893 8143 catalog_manager.cc:1347] Generated new cluster ID: 81ad2885d332458592ba3019bf4dfce9
I20260811 12:20:46.852043 8143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:46.878201 8143 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:46.880232 8143 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:46.892443 8143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8f276d7302040c6bed7867fb022ac96: Generated new TSK 0
I20260811 12:20:46.893428 8143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:46.907410 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8013
I20260811 12:20:46.909193 8104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:47.004480 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8013
W20260811 12:20:47.032913 7988 connection.cc:535] client connection to 127.0.0.1:33319 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33319): Connection reset by peer (error 104)
2026-08-11T12:20:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:47 dist-test-slave-rmhg krb5kdc[8176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:47 dist-test-slave-rmhg krb5kdc[8176](info): set up 2 sockets
Aug 11 12:20:47 dist-test-slave-rmhg krb5kdc[8176](info): commencing operation
krb5kdc: starting...
W20260811 12:20:48.270165 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:48 dist-test-slave-rmhg krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:20:48.608525 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33073
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:48.971069 8199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:48.971419 8199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:48.971572 8199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:48.979146 8199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:48.979306 8199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:48.979390 8199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:48.979475 8199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:48.979537 8199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:48.990688 8199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33073
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:48.992190 8199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:48.993937 8199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:49.007050 8209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:49.007035 8210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:49.007000 8213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:49.007783 8199 server_base.cc:1034] running on GCE node
I20260811 12:20:49.008690 8199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:49.010390 8199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:49.011660 8199 hybrid_clock.cc:648] HybridClock initialized: now 1786450849011602 us; error 54 us; skew 500 ppm
I20260811 12:20:49.012071 8199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:49 dist-test-slave-rmhg krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:49.015556 8199 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:20:49.017292 8199 webserver.cc:460] Webserver started at http://127.0.0.1:42977/ using document root <none> and password file <none>
I20260811 12:20:49.018031 8199 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:49.018141 8199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:49.018416 8199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:49.021708 8199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "66837d56856149b092e80f84a325b04d"
format_stamp: "Formatted at 2026-08-11 12:20:49 on dist-test-slave-rmhg"
server_key: "fc2a3c932bba464650d14db4b533ad4b"
server_key_iv: "5488b56d254fcb41c8d90b0be1adc7fc"
server_key_version: "encryptionkey@0"
I20260811 12:20:49.022646 8199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "66837d56856149b092e80f84a325b04d"
format_stamp: "Formatted at 2026-08-11 12:20:49 on dist-test-slave-rmhg"
server_key: "fc2a3c932bba464650d14db4b533ad4b"
server_key_iv: "5488b56d254fcb41c8d90b0be1adc7fc"
server_key_version: "encryptionkey@0"
I20260811 12:20:49.028419 8199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:20:49.032919 8223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:49.034539 8199 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:20:49.034771 8199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "66837d56856149b092e80f84a325b04d"
format_stamp: "Formatted at 2026-08-11 12:20:49 on dist-test-slave-rmhg"
server_key: "fc2a3c932bba464650d14db4b533ad4b"
server_key_iv: "5488b56d254fcb41c8d90b0be1adc7fc"
server_key_version: "encryptionkey@0"
I20260811 12:20:49.035002 8199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:49.073326 8199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:49.078416 8199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:49.078727 8199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:49.097575 8199 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33073
I20260811 12:20:49.097615 8298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33073 every 8 connection(s)
I20260811 12:20:49.099017 8199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:49.102845 8301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:49.103521 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8199
I20260811 12:20:49.103734 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:49.104332 7324 external_mini_cluster.cc:1428] Setting key d60016b901906c6c7afb679e9f198761
I20260811 12:20:49.117543 8301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d: Bootstrap starting.
I20260811 12:20:49.119256 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:49.106312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57321 (local address 127.0.0.1:33073)
0811 12:20:49.107128 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:49.107143 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:49.107190 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:20:49.109418 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:49.109447 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:49.109969 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:49.110360 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:49.111026 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:49.112417 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:49.114430 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:49.115204 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:49.115338 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:49.116743 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:49.116752 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:49.116769 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:49.117434 (+ 665us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:49.117521 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:49.117555 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:49.117705 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:49.117985 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 12:20:49.118173 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":377,"threads_started":1}
I20260811 12:20:49.121037 8301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:49.122325 8301 log.cc:824] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:49.123512 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:20:49.125964 8301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d: No bootstrap required, opened a new log
I20260811 12:20:49.135983 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:49.128928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48339 (local address 127.0.0.1:33073)
0811 12:20:49.129206 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:49.129212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:49.129562 (+ 350us) server_negotiation.cc:407] Connection header received
0811 12:20:49.129736 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:49.129740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:49.129809 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:49.129896 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:49.130519 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:49.131340 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:49.133406 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:49.133891 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:49.133999 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:49.135262 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:49.135267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:49.135270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:49.135290 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:49.135344 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:49.135360 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:49.135441 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:49.135618 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 12:20:49.135667 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:20:49.136329 8301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66837d56856149b092e80f84a325b04d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260811 12:20:49.136842 8301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:49.136969 8301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66837d56856149b092e80f84a325b04d, State: Initialized, Role: FOLLOWER
I20260811 12:20:49.137651 8301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66837d56856149b092e80f84a325b04d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260811 12:20:49.137880 8301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:49.137984 8301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:49.138109 8301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:49.140401 8301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66837d56856149b092e80f84a325b04d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260811 12:20:49.140910 8301 leader_election.cc:302] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66837d56856149b092e80f84a325b04d; no voters:
I20260811 12:20:49.141860 8301 leader_election.cc:288] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:49.141943 8313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:49.142706 8313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [term 1 LEADER]: Becoming Leader. State: Replica: 66837d56856149b092e80f84a325b04d, State: Running, Role: LEADER
I20260811 12:20:49.143353 8313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66837d56856149b092e80f84a325b04d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } }
I20260811 12:20:49.144061 8301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:49.146314 8317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66837d56856149b092e80f84a325b04d. Latest consensus state: current_term: 1 leader_uuid: "66837d56856149b092e80f84a325b04d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66837d56856149b092e80f84a325b04d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } } }
I20260811 12:20:49.146790 8317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:49.146692 8315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66837d56856149b092e80f84a325b04d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66837d56856149b092e80f84a325b04d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33073 } } }
I20260811 12:20:49.147032 8315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:49.149998 8331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:49.155784 8331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:49.164779 8331 catalog_manager.cc:1347] Generated new cluster ID: 4136498b88aa4e80a413015645e8d53e
I20260811 12:20:49.164914 8331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:49.174883 8331 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:49.176844 8331 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:49.190234 8331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66837d56856149b092e80f84a325b04d: Generated new TSK 0
I20260811 12:20:49.191182 8331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:49.202368 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8199
W20260811 12:20:49.203670 8292 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57321
W20260811 12:20:49.204665 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8199: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:20:49.204855 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8199
W20260811 12:20:49.231047 8171 connection.cc:535] client connection to 127.0.0.1:33073 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33073): Connection reset by peer (error 104)
2026-08-11T12:20:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2198 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:49 dist-test-slave-rmhg krb5kdc[8349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:49 dist-test-slave-rmhg krb5kdc[8349](info): set up 2 sockets
Aug 11 12:20:49 dist-test-slave-rmhg krb5kdc[8349](info): commencing operation
krb5kdc: starting...
W20260811 12:20:50.477550 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:50 dist-test-slave-rmhg krb5kdc[8349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:20:50.824363 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:51.182819 8369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:51.183140 8369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:51.183262 8369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:51.190769 8369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:51.190914 8369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:51.190968 8369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:51.191052 8369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:51.191112 8369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:51.201507 8369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37663
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:51.202920 8369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:51.204267 8369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:51.216351 8378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:51.216375 8384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:51.216410 8380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:51.217566 8369 server_base.cc:1034] running on GCE node
I20260811 12:20:51.218348 8369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:51.219799 8369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:51.221016 8369 hybrid_clock.cc:648] HybridClock initialized: now 1786450851220953 us; error 53 us; skew 500 ppm
I20260811 12:20:51.221402 8369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:51 dist-test-slave-rmhg krb5kdc[8349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:51.224575 8369 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:20:51.226279 8369 webserver.cc:460] Webserver started at http://127.0.0.1:39855/ using document root <none> and password file <none>
I20260811 12:20:51.227061 8369 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:51.227200 8369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:51.227532 8369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:51.230993 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "2461eedbe53142b3ad33161cc0f1ecfa"
format_stamp: "Formatted at 2026-08-11 12:20:51 on dist-test-slave-rmhg"
server_key: "1ad45ac1303bf4ee2003067da9625836"
server_key_iv: "13b2ba7a2607137c0fb3a24aaedceec7"
server_key_version: "encryptionkey@0"
I20260811 12:20:51.231927 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "2461eedbe53142b3ad33161cc0f1ecfa"
format_stamp: "Formatted at 2026-08-11 12:20:51 on dist-test-slave-rmhg"
server_key: "1ad45ac1303bf4ee2003067da9625836"
server_key_iv: "13b2ba7a2607137c0fb3a24aaedceec7"
server_key_version: "encryptionkey@0"
I20260811 12:20:51.237870 8369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 12:20:51.242213 8395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:51.243818 8369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:20:51.244022 8369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "2461eedbe53142b3ad33161cc0f1ecfa"
format_stamp: "Formatted at 2026-08-11 12:20:51 on dist-test-slave-rmhg"
server_key: "1ad45ac1303bf4ee2003067da9625836"
server_key_iv: "13b2ba7a2607137c0fb3a24aaedceec7"
server_key_version: "encryptionkey@0"
I20260811 12:20:51.244251 8369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:51.250703 8369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:51.255285 8369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:51.255604 8369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:51.275285 8369 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37663
I20260811 12:20:51.275336 8469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37663 every 8 connection(s)
I20260811 12:20:51.276835 8369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:51.278522 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8369
I20260811 12:20:51.278684 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:51.279137 7324 external_mini_cluster.cc:1428] Setting key 30fe70eb1a11dec40a292c578348721c
I20260811 12:20:51.281529 8472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:51.287999 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:51.281295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37665 (local address 127.0.0.1:37663)
0811 12:20:51.282183 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:51.282195 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:51.282251 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:20:51.283944 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:51.283972 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:51.284390 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:51.284799 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:51.286125 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:51.286137 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:51.286156 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:51.286177 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:51.286269 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:51.286320 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:51.286521 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:51.286667 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:20:51.286911 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":383,"threads_started":1}
I20260811 12:20:51.292331 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:20:51.294549 8472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa: Bootstrap starting.
I20260811 12:20:51.298200 8472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:51.299388 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:51.296934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40299 (local address 127.0.0.1:37663)
0811 12:20:51.297127 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:51.297133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:51.297456 (+ 323us) server_negotiation.cc:407] Connection header received
0811 12:20:51.297599 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:51.297604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:51.297667 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:51.297780 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:51.298742 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:51.298750 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:51.298753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:51.298771 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:51.298817 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:51.298833 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:51.298927 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:51.299064 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 12:20:51.299112 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:20:51.299665 8472 log.cc:824] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:51.302846 8472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa: No bootstrap required, opened a new log
I20260811 12:20:51.311223 8472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461eedbe53142b3ad33161cc0f1ecfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37663 } }
I20260811 12:20:51.311818 8472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:51.311955 8472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2461eedbe53142b3ad33161cc0f1ecfa, State: Initialized, Role: FOLLOWER
I20260811 12:20:51.312721 8472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461eedbe53142b3ad33161cc0f1ecfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37663 } }
I20260811 12:20:51.312980 8472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:51.313119 8472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:51.313337 8472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:51.315754 8472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461eedbe53142b3ad33161cc0f1ecfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37663 } }
I20260811 12:20:51.316335 8472 leader_election.cc:302] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2461eedbe53142b3ad33161cc0f1ecfa; no voters:
I20260811 12:20:51.316960 8472 leader_election.cc:288] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:51.317138 8488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:51.317853 8488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [term 1 LEADER]: Becoming Leader. State: Replica: 2461eedbe53142b3ad33161cc0f1ecfa, State: Running, Role: LEADER
I20260811 12:20:51.318403 8488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461eedbe53142b3ad33161cc0f1ecfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37663 } }
I20260811 12:20:51.319276 8472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:51.321396 8495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2461eedbe53142b3ad33161cc0f1ecfa. Latest consensus state: current_term: 1 leader_uuid: "2461eedbe53142b3ad33161cc0f1ecfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461eedbe53142b3ad33161cc0f1ecfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37663 } } }
I20260811 12:20:51.321357 8489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2461eedbe53142b3ad33161cc0f1ecfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461eedbe53142b3ad33161cc0f1ecfa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37663 } } }
I20260811 12:20:51.321921 8489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:51.321929 8495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:51.322908 8505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:51.332984 8505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:51.343087 8505 catalog_manager.cc:1347] Generated new cluster ID: 3a030b36f28e4786a82caee66cc5aa48
I20260811 12:20:51.343228 8505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:51.359357 8505 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:51.359848 8505 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:51.367525 8505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2461eedbe53142b3ad33161cc0f1ecfa: Generated new TSK 0
I20260811 12:20:51.368441 8505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:51.382565 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8369
W20260811 12:20:51.383890 8462 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37665
W20260811 12:20:51.384670 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8369: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:20:51.384820 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8369
2026-08-11T12:20:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2180 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:51 dist-test-slave-rmhg krb5kdc[8529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:51 dist-test-slave-rmhg krb5kdc[8529](info): set up 2 sockets
Aug 11 12:20:51 dist-test-slave-rmhg krb5kdc[8529](info): commencing operation
krb5kdc: starting...
W20260811 12:20:52.659955 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:52 dist-test-slave-rmhg krb5kdc[8529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:20:53.000599 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35113
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:53.361574 8554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:53.361897 8554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:53.362046 8554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:53.369642 8554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:53.369782 8554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:53.369858 8554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:53.369957 8554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:53.370018 8554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:53.380471 8554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:53.381959 8554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:53.383414 8554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:53.396476 8563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:53.396561 8564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:53.396517 8568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:53.397200 8554 server_base.cc:1034] running on GCE node
I20260811 12:20:53.398134 8554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:53.399786 8554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:53.401043 8554 hybrid_clock.cc:648] HybridClock initialized: now 1786450853400992 us; error 51 us; skew 500 ppm
I20260811 12:20:53.401448 8554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:53 dist-test-slave-rmhg krb5kdc[8529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:53.407835 8554 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:20:53.409574 8554 webserver.cc:460] Webserver started at http://127.0.0.1:40793/ using document root <none> and password file <none>
I20260811 12:20:53.410318 8554 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:53.410449 8554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:53.410789 8554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:53.414062 8554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "46fc85cb9ced45da945adf108d28b320"
format_stamp: "Formatted at 2026-08-11 12:20:53 on dist-test-slave-rmhg"
server_key: "5227e6a479be4c2e4283e00472bf2639"
server_key_iv: "39645c122aec21b093e8fcd686b0c508"
server_key_version: "encryptionkey@0"
I20260811 12:20:53.414877 8554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "46fc85cb9ced45da945adf108d28b320"
format_stamp: "Formatted at 2026-08-11 12:20:53 on dist-test-slave-rmhg"
server_key: "5227e6a479be4c2e4283e00472bf2639"
server_key_iv: "39645c122aec21b093e8fcd686b0c508"
server_key_version: "encryptionkey@0"
I20260811 12:20:53.420559 8554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 12:20:53.424401 8579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:53.426050 8554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:20:53.426242 8554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "46fc85cb9ced45da945adf108d28b320"
format_stamp: "Formatted at 2026-08-11 12:20:53 on dist-test-slave-rmhg"
server_key: "5227e6a479be4c2e4283e00472bf2639"
server_key_iv: "39645c122aec21b093e8fcd686b0c508"
server_key_version: "encryptionkey@0"
I20260811 12:20:53.426483 8554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:53.443308 8554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:53.448158 8554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:53.448473 8554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:53.466991 8554 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35113
I20260811 12:20:53.467058 8664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35113 every 8 connection(s)
I20260811 12:20:53.468525 8554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:53.472761 8665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:53.474866 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8554
I20260811 12:20:53.475076 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:53.475589 7324 external_mini_cluster.cc:1428] Setting key 780dcc8e5394660468a9ca2e58950c13
I20260811 12:20:53.486682 8665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320: Bootstrap starting.
I20260811 12:20:53.491006 8665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320: Neither blocks nor log segments found. Creating new log.
Aug 11 12:20:53 dist-test-slave-rmhg krb5kdc[8529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:20:53.492347 8665 log.cc:824] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:53.496166 8665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320: No bootstrap required, opened a new log
I20260811 12:20:53.501832 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:53.477692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58101 (local address 127.0.0.1:35113)
0811 12:20:53.478700 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:53.478718 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:53.478772 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:20:53.480609 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:53.480638 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:53.481124 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:53.481563 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:53.483587 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:53.485203 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:53.487274 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:53.488082 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:53.488261 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:53.493606 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:53.493621 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:53.493645 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:53.493713 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:53.496955 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:53.498001 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:53.498009 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:53.498017 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:53.498210 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:53.498752 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:53.498756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:53.498759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:53.499395 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:53.499656 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:53.500131 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 12:20:53.500567 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":438,"threads_started":1}
I20260811 12:20:53.505657 8665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc85cb9ced45da945adf108d28b320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35113 } }
I20260811 12:20:53.506120 8665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:53.506153 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:20:53.506239 8665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46fc85cb9ced45da945adf108d28b320, State: Initialized, Role: FOLLOWER
I20260811 12:20:53.506906 8665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc85cb9ced45da945adf108d28b320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35113 } }
I20260811 12:20:53.507202 8665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:53.507336 8665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:53.507495 8665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:53.510813 8665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc85cb9ced45da945adf108d28b320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35113 } }
I20260811 12:20:53.511479 8665 leader_election.cc:302] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46fc85cb9ced45da945adf108d28b320; no voters:
I20260811 12:20:53.512197 8665 leader_election.cc:288] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:53.512322 8684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:53.513757 8684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [term 1 LEADER]: Becoming Leader. State: Replica: 46fc85cb9ced45da945adf108d28b320, State: Running, Role: LEADER
I20260811 12:20:53.514564 8684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc85cb9ced45da945adf108d28b320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35113 } }
I20260811 12:20:53.515208 8665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:53.518082 8687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46fc85cb9ced45da945adf108d28b320. Latest consensus state: current_term: 1 leader_uuid: "46fc85cb9ced45da945adf108d28b320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc85cb9ced45da945adf108d28b320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35113 } } }
I20260811 12:20:53.518606 8687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:53.519816 8685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46fc85cb9ced45da945adf108d28b320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fc85cb9ced45da945adf108d28b320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35113 } } }
I20260811 12:20:53.520143 8685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:53.521456 8691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:53.528524 8691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:53.528666 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:53.511567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37637 (local address 127.0.0.1:35113)
0811 12:20:53.511773 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:53.511780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:53.512212 (+ 432us) server_negotiation.cc:407] Connection header received
0811 12:20:53.512435 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:53.512441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:53.512531 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:53.512649 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:53.514887 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:53.515754 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:53.517405 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:53.518102 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:53.518222 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:53.521586 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:53.521593 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:53.521598 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:20:53.521636 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:53.525109 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:53.526170 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:53.526176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:53.526179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:53.526372 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:20:53.526872 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:20:53.526878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:20:53.526882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:20:53.527353 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:53.527613 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:53.528035 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 12:20:53.528285 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:20:53.540442 8691 catalog_manager.cc:1347] Generated new cluster ID: 280355dffedc4f20987dbb5b4d68eefe
I20260811 12:20:53.540606 8691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:53.574368 8691 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:53.575838 8691 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:53.590476 8691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46fc85cb9ced45da945adf108d28b320: Generated new TSK 0
I20260811 12:20:53.591502 8691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:53.602782 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8554
I20260811 12:20:53.604022 8657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:20:53.701893 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8554
2026-08-11T12:20:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:53 dist-test-slave-rmhg krb5kdc[8723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:53 dist-test-slave-rmhg krb5kdc[8723](info): set up 2 sockets
Aug 11 12:20:53 dist-test-slave-rmhg krb5kdc[8723](info): commencing operation
krb5kdc: starting...
W20260811 12:20:54.971788 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:55 dist-test-slave-rmhg krb5kdc[8723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:20:55.315984 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:55.672696 8747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:55.673010 8747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:55.673136 8747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:55.680581 8747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:55.680724 8747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:55.680799 8747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:55.680886 8747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:55.680948 8747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:55.691337 8747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:55.692903 8747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:55.694648 8747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:55.706933 8756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:55.706974 8759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:55.707012 8761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:55.707870 8747 server_base.cc:1034] running on GCE node
I20260811 12:20:55.708712 8747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:55.710274 8747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:55.711488 8747 hybrid_clock.cc:648] HybridClock initialized: now 1786450855711433 us; error 49 us; skew 500 ppm
I20260811 12:20:55.711849 8747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:55 dist-test-slave-rmhg krb5kdc[8723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450855, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:55.715097 8747 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:20:55.716670 8747 webserver.cc:460] Webserver started at http://127.0.0.1:42467/ using document root <none> and password file <none>
I20260811 12:20:55.717470 8747 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:55.717602 8747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:55.717921 8747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:55.720999 8747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "1c7996e81268425ab1c6ba2115f29e57"
format_stamp: "Formatted at 2026-08-11 12:20:55 on dist-test-slave-rmhg"
server_key: "7feab1a0aea121eb5628c229169c27a3"
server_key_iv: "53636cf153126cd786806e6531efacd0"
server_key_version: "encryptionkey@0"
I20260811 12:20:55.721961 8747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "1c7996e81268425ab1c6ba2115f29e57"
format_stamp: "Formatted at 2026-08-11 12:20:55 on dist-test-slave-rmhg"
server_key: "7feab1a0aea121eb5628c229169c27a3"
server_key_iv: "53636cf153126cd786806e6531efacd0"
server_key_version: "encryptionkey@0"
I20260811 12:20:55.727511 8747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:20:55.731235 8769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:55.732651 8747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:20:55.732831 8747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "1c7996e81268425ab1c6ba2115f29e57"
format_stamp: "Formatted at 2026-08-11 12:20:55 on dist-test-slave-rmhg"
server_key: "7feab1a0aea121eb5628c229169c27a3"
server_key_iv: "53636cf153126cd786806e6531efacd0"
server_key_version: "encryptionkey@0"
I20260811 12:20:55.733060 8747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:55.752372 8747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:55.757334 8747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:55.757632 8747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:55.774912 8747 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44069
I20260811 12:20:55.774935 8861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44069 every 8 connection(s)
I20260811 12:20:55.776350 8747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:55.780105 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8747
I20260811 12:20:55.780288 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:55.780409 8863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:55.780768 7324 external_mini_cluster.cc:1428] Setting key 55c09b8a848b0bc17c02e8033cb60d89
I20260811 12:20:55.796635 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:55.782726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33693 (local address 127.0.0.1:44069)
0811 12:20:55.783635 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:55.783653 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:55.783722 (+ 69us) server_negotiation.cc:407] Connection header received
0811 12:20:55.785703 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:55.785739 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:55.786335 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:55.786857 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:55.787447 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:55.789550 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:55.790956 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:55.792045 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:55.792253 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:55.793482 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:55.793494 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:55.793516 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:55.794338 (+ 822us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:55.794504 (+ 166us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:55.794539 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:55.794657 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:55.794899 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 12:20:55.795213 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":348,"threads_started":1}
I20260811 12:20:55.796665 8863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57: Bootstrap starting.
I20260811 12:20:55.800577 8863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:55.801801 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:20:55.801970 8863 log.cc:824] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:55.809641 8863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57: No bootstrap required, opened a new log
I20260811 12:20:55.818568 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:55.811196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38291 (local address 127.0.0.1:44069)
0811 12:20:55.811402 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:55.811410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:55.811912 (+ 502us) server_negotiation.cc:407] Connection header received
0811 12:20:55.812091 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:55.812095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:55.812174 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:55.812271 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:55.812918 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:55.813730 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:55.815788 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:20:55.816320 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:20:55.816429 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:20:55.817739 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:55.817748 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:55.817752 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:55.817772 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:55.817823 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:55.817843 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:55.817929 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:55.818088 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 12:20:55.818184 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:20:55.819963 8863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7996e81268425ab1c6ba2115f29e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44069 } }
I20260811 12:20:55.820408 8863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:55.820513 8863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7996e81268425ab1c6ba2115f29e57, State: Initialized, Role: FOLLOWER
I20260811 12:20:55.821133 8863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7996e81268425ab1c6ba2115f29e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44069 } }
I20260811 12:20:55.821444 8863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:55.821587 8863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:55.821758 8863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:55.824071 8863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7996e81268425ab1c6ba2115f29e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44069 } }
I20260811 12:20:55.824674 8863 leader_election.cc:302] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c7996e81268425ab1c6ba2115f29e57; no voters:
I20260811 12:20:55.825310 8863 leader_election.cc:288] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:55.825440 8882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:55.827216 8882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [term 1 LEADER]: Becoming Leader. State: Replica: 1c7996e81268425ab1c6ba2115f29e57, State: Running, Role: LEADER
I20260811 12:20:55.827550 8863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:55.827798 8882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7996e81268425ab1c6ba2115f29e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44069 } }
I20260811 12:20:55.830595 8883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c7996e81268425ab1c6ba2115f29e57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7996e81268425ab1c6ba2115f29e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44069 } } }
I20260811 12:20:55.831050 8883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:55.831799 8885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c7996e81268425ab1c6ba2115f29e57. Latest consensus state: current_term: 1 leader_uuid: "1c7996e81268425ab1c6ba2115f29e57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7996e81268425ab1c6ba2115f29e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44069 } } }
I20260811 12:20:55.832119 8896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:55.832191 8885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:55.843569 8896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:55.854621 8896 catalog_manager.cc:1347] Generated new cluster ID: 8c8ca6ace71a453b9bd2a3cd4d8d6a71
I20260811 12:20:55.854779 8896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:55.868568 8896 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:55.870194 8896 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:55.875965 8896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c7996e81268425ab1c6ba2115f29e57: Generated new TSK 0
I20260811 12:20:55.877192 8896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:55.892944 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8747
W20260811 12:20:55.894269 8853 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33693
W20260811 12:20:55.895121 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8747: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:20:55.895300 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8747
2026-08-11T12:20:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2198 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:56 dist-test-slave-rmhg krb5kdc[8916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:56 dist-test-slave-rmhg krb5kdc[8916](info): set up 2 sockets
Aug 11 12:20:56 dist-test-slave-rmhg krb5kdc[8916](info): commencing operation
krb5kdc: starting...
W20260811 12:20:57.181931 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:57 dist-test-slave-rmhg krb5kdc[8916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:20:57.524886 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35455
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:20:57.891578 8936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:20:57.891911 8936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:20:57.892007 8936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:20:57.899669 8936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:20:57.899811 8936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:20:57.899860 8936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:20:57.899925 8936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:20:57.899971 8936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:20:57.910864 8936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:20:57.912241 8936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:20:57.913750 8936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:20:57.926296 8943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:57.926383 8946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:20:57.926256 8949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:20:57.927532 8936 server_base.cc:1034] running on GCE node
I20260811 12:20:57.928501 8936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:20:57.930147 8936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:20:57.931418 8936 hybrid_clock.cc:648] HybridClock initialized: now 1786450857931357 us; error 52 us; skew 500 ppm
I20260811 12:20:57.931859 8936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:20:57 dist-test-slave-rmhg krb5kdc[8916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:20:57.935297 8936 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:20:57.936992 8936 webserver.cc:460] Webserver started at http://127.0.0.1:45725/ using document root <none> and password file <none>
I20260811 12:20:57.937856 8936 fs_manager.cc:359] Metadata directory not provided
I20260811 12:20:57.937984 8936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:20:57.938350 8936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:20:57.941658 8936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "ed0791e4924f41bda8ffca0c8e0fa692"
format_stamp: "Formatted at 2026-08-11 12:20:57 on dist-test-slave-rmhg"
server_key: "739771182fcfeb28814987032409d58e"
server_key_iv: "472799d455b84e5ade674bf7902057e6"
server_key_version: "encryptionkey@0"
I20260811 12:20:57.942507 8936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "ed0791e4924f41bda8ffca0c8e0fa692"
format_stamp: "Formatted at 2026-08-11 12:20:57 on dist-test-slave-rmhg"
server_key: "739771182fcfeb28814987032409d58e"
server_key_iv: "472799d455b84e5ade674bf7902057e6"
server_key_version: "encryptionkey@0"
I20260811 12:20:57.948463 8936 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 12:20:57.952728 8959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:20:57.954355 8936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:20:57.954552 8936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "ed0791e4924f41bda8ffca0c8e0fa692"
format_stamp: "Formatted at 2026-08-11 12:20:57 on dist-test-slave-rmhg"
server_key: "739771182fcfeb28814987032409d58e"
server_key_iv: "472799d455b84e5ade674bf7902057e6"
server_key_version: "encryptionkey@0"
I20260811 12:20:57.954800 8936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:20:57.961575 8936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:20:57.966099 8936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:20:57.966408 8936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:20:57.985520 8936 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35455
I20260811 12:20:57.985569 9039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35455 every 8 connection(s)
I20260811 12:20:57.987036 8936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:20:57.989681 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 8936
I20260811 12:20:57.989885 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:20:57.990482 7324 external_mini_cluster.cc:1428] Setting key 59bd5b3205e5c102ab63ad290e23ffa4
I20260811 12:20:57.991770 9040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:20:57.999140 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:57.992522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37887 (local address 127.0.0.1:35455)
0811 12:20:57.993447 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:57.993460 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:57.993508 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:20:57.995157 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:57.995192 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:57.995618 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:57.996016 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:57.997302 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:57.997309 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:57.997325 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:57.997350 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:57.997437 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:57.997469 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:57.997687 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:57.997797 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 12:20:57.998016 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":405,"threads_started":1}
I20260811 12:20:58.003309 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:20:58.006376 9040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692: Bootstrap starting.
I20260811 12:20:58.010946 9040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692: Neither blocks nor log segments found. Creating new log.
I20260811 12:20:58.011756 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:20:58.009146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38001 (local address 127.0.0.1:35455)
0811 12:20:58.009387 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 12:20:58.009392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:20:58.009703 (+ 311us) server_negotiation.cc:407] Connection header received
0811 12:20:58.009904 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:20:58.009908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:20:58.009963 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:20:58.010052 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:20:58.011061 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:20:58.011067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:20:58.011072 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:20:58.011118 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:20:58.011167 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:20:58.011184 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:20:58.011274 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:20:58.011425 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 12:20:58.011480 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 12:20:58.012486 9040 log.cc:824] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692: Log is configured to *not* fsync() on all Append() calls
I20260811 12:20:58.015710 9040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692: No bootstrap required, opened a new log
I20260811 12:20:58.023561 9040 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0791e4924f41bda8ffca0c8e0fa692" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } }
I20260811 12:20:58.024127 9040 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:20:58.024246 9040 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0791e4924f41bda8ffca0c8e0fa692, State: Initialized, Role: FOLLOWER
I20260811 12:20:58.024919 9040 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0791e4924f41bda8ffca0c8e0fa692" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } }
I20260811 12:20:58.025182 9040 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:20:58.025326 9040 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:20:58.025487 9040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:20:58.027896 9040 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0791e4924f41bda8ffca0c8e0fa692" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } }
I20260811 12:20:58.028426 9040 leader_election.cc:302] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed0791e4924f41bda8ffca0c8e0fa692; no voters:
I20260811 12:20:58.029021 9040 leader_election.cc:288] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:20:58.029209 9056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:20:58.029927 9056 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [term 1 LEADER]: Becoming Leader. State: Replica: ed0791e4924f41bda8ffca0c8e0fa692, State: Running, Role: LEADER
I20260811 12:20:58.030534 9056 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0791e4924f41bda8ffca0c8e0fa692" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } }
I20260811 12:20:58.031284 9040 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:20:58.033711 9060 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed0791e4924f41bda8ffca0c8e0fa692. Latest consensus state: current_term: 1 leader_uuid: "ed0791e4924f41bda8ffca0c8e0fa692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0791e4924f41bda8ffca0c8e0fa692" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } } }
I20260811 12:20:58.034065 9060 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:58.034574 9057 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed0791e4924f41bda8ffca0c8e0fa692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0791e4924f41bda8ffca0c8e0fa692" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } } }
I20260811 12:20:58.034893 9057 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692 [sys.catalog]: This master's current role is: LEADER
I20260811 12:20:58.037477 9068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:20:58.045372 9068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:20:58.055279 9068 catalog_manager.cc:1347] Generated new cluster ID: d7a1d1769b4e4df48d6db45ad7b8b0fc
I20260811 12:20:58.055436 9068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:20:58.076386 9068 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:20:58.076898 9068 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:20:58.094290 9068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed0791e4924f41bda8ffca0c8e0fa692: Generated new TSK 0
I20260811 12:20:58.095252 9068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:20:58.111464 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8936
W20260811 12:20:58.112730 9034 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37887
W20260811 12:20:58.113541 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 8936: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:20:58.113729 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 8936
2026-08-11T12:20:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2215 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:20:58 dist-test-slave-rmhg krb5kdc[9092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:20:58 dist-test-slave-rmhg krb5kdc[9092](info): set up 2 sockets
Aug 11 12:20:58 dist-test-slave-rmhg krb5kdc[9092](info): commencing operation
krb5kdc: starting...
W20260811 12:20:59.499192 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:20:59 dist-test-slave-rmhg krb5kdc[9092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:20:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:20:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:20:59.848362 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:00.206943 9108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:00.207259 9108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:00.207396 9108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:00.214875 9108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:00.215011 9108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:00.215070 9108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:00.215157 9108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:00.215210 9108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:00.225455 9108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:00.226871 9108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:00.228241 9108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:00.240674 9114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:00.240538 9121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:00.240597 9116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:00.241855 9108 server_base.cc:1034] running on GCE node
I20260811 12:21:00.242863 9108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:00.244572 9108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:00.245805 9108 hybrid_clock.cc:648] HybridClock initialized: now 1786450860245735 us; error 61 us; skew 500 ppm
I20260811 12:21:00.246215 9108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:00 dist-test-slave-rmhg krb5kdc[9092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:00.254298 9108 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:21:00.256070 9108 webserver.cc:460] Webserver started at http://127.0.0.1:43339/ using document root <none> and password file <none>
I20260811 12:21:00.256848 9108 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:00.256981 9108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:00.257332 9108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:00.260516 9108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "e6fd328b984d4ffbb0a179fd54466046"
format_stamp: "Formatted at 2026-08-11 12:21:00 on dist-test-slave-rmhg"
server_key: "b10c5fd4c9c3e6d29e66a6d4576ae323"
server_key_iv: "5fe6780cd55631bc532923e74131c09d"
server_key_version: "encryptionkey@0"
I20260811 12:21:00.261384 9108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "e6fd328b984d4ffbb0a179fd54466046"
format_stamp: "Formatted at 2026-08-11 12:21:00 on dist-test-slave-rmhg"
server_key: "b10c5fd4c9c3e6d29e66a6d4576ae323"
server_key_iv: "5fe6780cd55631bc532923e74131c09d"
server_key_version: "encryptionkey@0"
I20260811 12:21:00.267207 9108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:21:00.271334 9133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:00.272877 9108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:21:00.273099 9108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "e6fd328b984d4ffbb0a179fd54466046"
format_stamp: "Formatted at 2026-08-11 12:21:00 on dist-test-slave-rmhg"
server_key: "b10c5fd4c9c3e6d29e66a6d4576ae323"
server_key_iv: "5fe6780cd55631bc532923e74131c09d"
server_key_version: "encryptionkey@0"
I20260811 12:21:00.273430 9108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:00.305068 9108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:00.309480 9108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:00.309746 9108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:00.328833 9108 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46017
I20260811 12:21:00.328871 9224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46017 every 8 connection(s)
I20260811 12:21:00.330325 9108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:00.334407 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9108
I20260811 12:21:00.334626 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:00.334755 9225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:00.335237 7324 external_mini_cluster.cc:1428] Setting key 9b2675fee3e9ccf8b44c8cfe7d40c909
I20260811 12:21:00.350769 9225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046: Bootstrap starting.
Aug 11 12:21:00 dist-test-slave-rmhg krb5kdc[9092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:21:00.354869 9225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:00.356133 9225 log.cc:824] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:00.360244 9225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046: No bootstrap required, opened a new log
I20260811 12:21:00.363009 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:00.337446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:35341 (local address 127.0.0.1:46017)
0811 12:21:00.338351 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:00.338366 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:00.338411 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:21:00.340184 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:00.340212 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:00.340703 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:00.341078 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:00.343122 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:00.344586 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:00.346617 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:00.347484 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:00.347658 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:00.353489 (+ 5831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:00.353501 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:00.353521 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:00.353585 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:00.357049 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:00.358333 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:00.358342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:00.358351 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:00.358576 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:00.359286 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:00.359292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:00.359295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:00.360117 (+ 822us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:00.360497 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:00.361026 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 12:21:00.361469 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":412,"threads_started":1}
I20260811 12:21:00.367578 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:00.370007 9225 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fd328b984d4ffbb0a179fd54466046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46017 } }
I20260811 12:21:00.370455 9225 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:00.370625 9225 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6fd328b984d4ffbb0a179fd54466046, State: Initialized, Role: FOLLOWER
I20260811 12:21:00.371285 9225 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fd328b984d4ffbb0a179fd54466046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46017 } }
I20260811 12:21:00.371537 9225 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:00.371688 9225 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:00.371901 9225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:00.374497 9225 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fd328b984d4ffbb0a179fd54466046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46017 } }
I20260811 12:21:00.375069 9225 leader_election.cc:302] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6fd328b984d4ffbb0a179fd54466046; no voters:
I20260811 12:21:00.375859 9225 leader_election.cc:288] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:00.376077 9240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:00.382879 9240 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [term 1 LEADER]: Becoming Leader. State: Replica: e6fd328b984d4ffbb0a179fd54466046, State: Running, Role: LEADER
I20260811 12:21:00.383883 9240 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fd328b984d4ffbb0a179fd54466046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46017 } }
I20260811 12:21:00.384406 9225 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:00.386884 9242 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6fd328b984d4ffbb0a179fd54466046. Latest consensus state: current_term: 1 leader_uuid: "e6fd328b984d4ffbb0a179fd54466046" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fd328b984d4ffbb0a179fd54466046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46017 } } }
I20260811 12:21:00.387351 9242 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:00.388367 9241 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6fd328b984d4ffbb0a179fd54466046" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fd328b984d4ffbb0a179fd54466046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46017 } } }
I20260811 12:21:00.388697 9241 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:00.391305 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:00.372149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:35339 (local address 127.0.0.1:46017)
0811 12:21:00.372331 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:00.372339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:00.372760 (+ 421us) server_negotiation.cc:407] Connection header received
0811 12:21:00.372952 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:00.372958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:00.373049 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:00.373141 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:00.374614 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:00.375627 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:00.377611 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:00.378304 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:00.378412 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:00.382112 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:00.382122 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:00.382127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:00.382175 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:00.386364 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:00.387512 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:00.387517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:00.387520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:00.387677 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:00.388347 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:00.388356 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:00.388360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:00.388824 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:00.390461 (+ 1637us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:00.390670 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 12:21:00.390891 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:21:00.397909 9245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:00.403606 9245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:00.414016 9245 catalog_manager.cc:1347] Generated new cluster ID: 9ca95c0143fe4661b9a90fc0b5302708
I20260811 12:21:00.414151 9245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:00.441809 9245 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:00.443364 9245 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:00.449827 9245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6fd328b984d4ffbb0a179fd54466046: Generated new TSK 0
I20260811 12:21:00.450771 9245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:00.461369 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9108
I20260811 12:21:00.462678 9218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:00.559531 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9108
W20260811 12:21:00.586812 9089 connection.cc:535] client connection to 127.0.0.1:46017 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46017): Connection reset by peer (error 104)
2026-08-11T12:21:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:00 dist-test-slave-rmhg krb5kdc[9272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:00 dist-test-slave-rmhg krb5kdc[9272](info): set up 2 sockets
Aug 11 12:21:00 dist-test-slave-rmhg krb5kdc[9272](info): commencing operation
krb5kdc: starting...
W20260811 12:21:01.836848 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:02 dist-test-slave-rmhg krb5kdc[9272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:21:02.189216 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35625
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:02.559300 9291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:02.559635 9291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:02.559773 9291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:02.567444 9291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:02.567595 9291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:02.567662 9291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:02.567760 9291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:02.567811 9291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:02.578233 9291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:02.579700 9291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:02.581187 9291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:02.594015 9298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:02.593989 9300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:02.593940 9302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:02.594878 9291 server_base.cc:1034] running on GCE node
I20260811 12:21:02.595752 9291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:02.597373 9291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:02.598598 9291 hybrid_clock.cc:648] HybridClock initialized: now 1786450862598544 us; error 53 us; skew 500 ppm
I20260811 12:21:02.598989 9291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:02 dist-test-slave-rmhg krb5kdc[9272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450862, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:02.602399 9291 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:21:02.604136 9291 webserver.cc:460] Webserver started at http://127.0.0.1:45043/ using document root <none> and password file <none>
I20260811 12:21:02.604931 9291 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:02.605060 9291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:02.605429 9291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:02.608675 9291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "8f59975b52994eadb98dd2805f941dc0"
format_stamp: "Formatted at 2026-08-11 12:21:02 on dist-test-slave-rmhg"
server_key: "c131c516b077854f3b4d79451501fdbf"
server_key_iv: "8361e11aff737b43b064c2541d443c18"
server_key_version: "encryptionkey@0"
I20260811 12:21:02.609539 9291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "8f59975b52994eadb98dd2805f941dc0"
format_stamp: "Formatted at 2026-08-11 12:21:02 on dist-test-slave-rmhg"
server_key: "c131c516b077854f3b4d79451501fdbf"
server_key_iv: "8361e11aff737b43b064c2541d443c18"
server_key_version: "encryptionkey@0"
I20260811 12:21:02.615072 9291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:21:02.618752 9312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:02.620173 9291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:21:02.620348 9291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "8f59975b52994eadb98dd2805f941dc0"
format_stamp: "Formatted at 2026-08-11 12:21:02 on dist-test-slave-rmhg"
server_key: "c131c516b077854f3b4d79451501fdbf"
server_key_iv: "8361e11aff737b43b064c2541d443c18"
server_key_version: "encryptionkey@0"
I20260811 12:21:02.620544 9291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:02.637516 9291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:02.642159 9291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:02.642462 9291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:02.661213 9291 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35625
I20260811 12:21:02.661239 9393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35625 every 8 connection(s)
I20260811 12:21:02.662703 9291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:02.666298 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9291
I20260811 12:21:02.666525 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:02.667102 7324 external_mini_cluster.cc:1428] Setting key eb1bef3c9a5daf651167536f3f2bd795
I20260811 12:21:02.667189 9394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:02.681928 9394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0: Bootstrap starting.
I20260811 12:21:02.682097 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:02.669105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:39163 (local address 127.0.0.1:35625)
0811 12:21:02.669946 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:02.669959 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:02.670008 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:21:02.672111 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:02.672138 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:02.672590 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:02.672984 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:02.673745 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:02.675144 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:02.677046 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:02.677833 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:02.677983 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:02.679414 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:02.679422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:02.679439 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:02.680079 (+ 640us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:02.680167 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:02.680197 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:02.680330 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:02.680622 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 12:21:02.680851 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":337,"threads_started":1}
I20260811 12:21:02.685474 9394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:02.686260 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:02.686712 9394 log.cc:824] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:02.690374 9394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0: No bootstrap required, opened a new log
I20260811 12:21:02.698390 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:02.691388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:51423 (local address 127.0.0.1:35625)
0811 12:21:02.691608 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:02.691614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:02.691997 (+ 383us) server_negotiation.cc:407] Connection header received
0811 12:21:02.692175 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:02.692181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:02.692274 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:02.692398 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:02.692885 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:02.694158 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:02.695605 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:02.696307 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:02.696469 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:02.697584 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:02.697591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:02.697596 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:02.697621 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:02.697688 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:02.697708 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:02.697819 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:02.697969 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 12:21:02.698034 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:21:02.700417 9394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f59975b52994eadb98dd2805f941dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35625 } }
I20260811 12:21:02.700850 9394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:02.700934 9394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f59975b52994eadb98dd2805f941dc0, State: Initialized, Role: FOLLOWER
I20260811 12:21:02.701655 9394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f59975b52994eadb98dd2805f941dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35625 } }
I20260811 12:21:02.702028 9394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:02.702172 9394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:02.702371 9394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:02.704720 9394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f59975b52994eadb98dd2805f941dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35625 } }
I20260811 12:21:02.705340 9394 leader_election.cc:302] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f59975b52994eadb98dd2805f941dc0; no voters:
I20260811 12:21:02.705926 9394 leader_election.cc:288] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:02.706074 9408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:02.707232 9408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [term 1 LEADER]: Becoming Leader. State: Replica: 8f59975b52994eadb98dd2805f941dc0, State: Running, Role: LEADER
I20260811 12:21:02.707849 9408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f59975b52994eadb98dd2805f941dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35625 } }
I20260811 12:21:02.708351 9394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:02.710925 9410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f59975b52994eadb98dd2805f941dc0. Latest consensus state: current_term: 1 leader_uuid: "8f59975b52994eadb98dd2805f941dc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f59975b52994eadb98dd2805f941dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35625 } } }
I20260811 12:21:02.711385 9410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:02.712469 9424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:02.712411 9409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f59975b52994eadb98dd2805f941dc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f59975b52994eadb98dd2805f941dc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35625 } } }
I20260811 12:21:02.712774 9409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:02.719089 9424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:02.730640 9424 catalog_manager.cc:1347] Generated new cluster ID: 9b306300e3064cffb9bafac92cc9556f
I20260811 12:21:02.730779 9424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:02.752990 9424 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:02.754719 9424 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:02.763479 9424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f59975b52994eadb98dd2805f941dc0: Generated new TSK 0
I20260811 12:21:02.764441 9424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:02.770756 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9291
W20260811 12:21:02.772583 9388 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.204:39163
W20260811 12:21:02.773466 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9291: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:21:02.773635 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9291
W20260811 12:21:02.808308 9270 connection.cc:535] client connection to 127.0.0.1:35625 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35625): Connection reset by peer (error 104)
2026-08-11T12:21:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2221 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:02 dist-test-slave-rmhg krb5kdc[9450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:02 dist-test-slave-rmhg krb5kdc[9450](info): set up 2 sockets
Aug 11 12:21:02 dist-test-slave-rmhg krb5kdc[9450](info): commencing operation
krb5kdc: starting...
W20260811 12:21:04.048806 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:04 dist-test-slave-rmhg krb5kdc[9450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:21:04.406194 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45473
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:04.774367 9469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:04.774724 9469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:04.774868 9469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:04.782858 9469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:04.783030 9469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:04.783119 9469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:04.783203 9469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:04.783264 9469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:04.793926 9469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:04.795363 9469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:04.796854 9469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:04.809700 9478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:04.809700 9480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:04.809737 9483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:04.810684 9469 server_base.cc:1034] running on GCE node
I20260811 12:21:04.811599 9469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:04.813241 9469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:04.814486 9469 hybrid_clock.cc:648] HybridClock initialized: now 1786450864814428 us; error 49 us; skew 500 ppm
I20260811 12:21:04.814862 9469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:04 dist-test-slave-rmhg krb5kdc[9450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:04.818447 9469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:21:04.820154 9469 webserver.cc:460] Webserver started at http://127.0.0.1:40335/ using document root <none> and password file <none>
I20260811 12:21:04.820950 9469 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:04.821084 9469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:04.821422 9469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:04.824712 9469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "1aab8aef47614cf08e3e085391ae27e2"
format_stamp: "Formatted at 2026-08-11 12:21:04 on dist-test-slave-rmhg"
server_key: "7e7d79dc48937cae226cfbf5a6d24df9"
server_key_iv: "105e9eea02a88ee05dcd78a29eb2f44f"
server_key_version: "encryptionkey@0"
I20260811 12:21:04.825593 9469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "1aab8aef47614cf08e3e085391ae27e2"
format_stamp: "Formatted at 2026-08-11 12:21:04 on dist-test-slave-rmhg"
server_key: "7e7d79dc48937cae226cfbf5a6d24df9"
server_key_iv: "105e9eea02a88ee05dcd78a29eb2f44f"
server_key_version: "encryptionkey@0"
I20260811 12:21:04.831995 9469 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 12:21:04.835958 9500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:04.837548 9469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:21:04.837774 9469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "1aab8aef47614cf08e3e085391ae27e2"
format_stamp: "Formatted at 2026-08-11 12:21:04 on dist-test-slave-rmhg"
server_key: "7e7d79dc48937cae226cfbf5a6d24df9"
server_key_iv: "105e9eea02a88ee05dcd78a29eb2f44f"
server_key_version: "encryptionkey@0"
I20260811 12:21:04.838002 9469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:04.844471 9469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:04.848986 9469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:04.849337 9469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:04.867224 9469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45473
I20260811 12:21:04.867265 9581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45473 every 8 connection(s)
I20260811 12:21:04.868783 9469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:04.872225 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9469
I20260811 12:21:04.872401 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:04.872895 7324 external_mini_cluster.cc:1428] Setting key 545753f662b956840846d1df8cf867d3
I20260811 12:21:04.873188 9582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:04.882893 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:04.874951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:38945 (local address 127.0.0.1:45473)
0811 12:21:04.875867 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:04.875884 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:04.875944 (+ 60us) server_negotiation.cc:407] Connection header received
0811 12:21:04.878368 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:04.878408 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:04.878974 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:04.879526 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:04.880697 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:04.880707 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:04.880726 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:04.880746 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:04.880832 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:04.880862 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:04.881122 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:04.881293 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 12:21:04.881539 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":376,"threads_started":1}
I20260811 12:21:04.887111 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:04.888844 9582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2: Bootstrap starting.
I20260811 12:21:04.893368 9582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:04.894888 9582 log.cc:824] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:04.895294 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:04.892864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:34847 (local address 127.0.0.1:45473)
0811 12:21:04.893061 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:04.893066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:04.893330 (+ 264us) server_negotiation.cc:407] Connection header received
0811 12:21:04.893494 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:04.893499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:04.893577 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:04.893709 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:04.894662 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:04.894668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:04.894673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:04.894695 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:04.894747 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:04.894767 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:04.894904 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:04.894954 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 12:21:04.895022 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:21:04.898317 9582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2: No bootstrap required, opened a new log
I20260811 12:21:04.906251 9582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aab8aef47614cf08e3e085391ae27e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } }
I20260811 12:21:04.906708 9582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:04.906823 9582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aab8aef47614cf08e3e085391ae27e2, State: Initialized, Role: FOLLOWER
I20260811 12:21:04.907550 9582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aab8aef47614cf08e3e085391ae27e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } }
I20260811 12:21:04.907832 9582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:04.908010 9582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:04.908195 9582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:04.911008 9582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aab8aef47614cf08e3e085391ae27e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } }
I20260811 12:21:04.911609 9582 leader_election.cc:302] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aab8aef47614cf08e3e085391ae27e2; no voters:
I20260811 12:21:04.912369 9582 leader_election.cc:288] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:04.912492 9599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:04.913189 9599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [term 1 LEADER]: Becoming Leader. State: Replica: 1aab8aef47614cf08e3e085391ae27e2, State: Running, Role: LEADER
I20260811 12:21:04.913895 9599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aab8aef47614cf08e3e085391ae27e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } }
I20260811 12:21:04.914983 9582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:04.917109 9602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aab8aef47614cf08e3e085391ae27e2. Latest consensus state: current_term: 1 leader_uuid: "1aab8aef47614cf08e3e085391ae27e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aab8aef47614cf08e3e085391ae27e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } } }
I20260811 12:21:04.917212 9601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aab8aef47614cf08e3e085391ae27e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aab8aef47614cf08e3e085391ae27e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } } }
I20260811 12:21:04.917673 9601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:04.919701 9602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:04.920020 9607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:04.926331 9607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:04.936890 9607 catalog_manager.cc:1347] Generated new cluster ID: f5adf4c0f6184144bfc039bd3da34e07
I20260811 12:21:04.937026 9607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:04.948995 9607 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:04.949508 9607 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:04.959734 9607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aab8aef47614cf08e3e085391ae27e2: Generated new TSK 0
I20260811 12:21:04.960708 9607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:04.966787 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9469
W20260811 12:21:04.968551 9574 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.204:38945
W20260811 12:21:04.969372 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9469: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:21:04.969515 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9469
2026-08-11T12:21:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2189 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:05 dist-test-slave-rmhg krb5kdc[9629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:05 dist-test-slave-rmhg krb5kdc[9629](info): set up 2 sockets
Aug 11 12:21:05 dist-test-slave-rmhg krb5kdc[9629](info): commencing operation
krb5kdc: starting...
W20260811 12:21:06.252285 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:06 dist-test-slave-rmhg krb5kdc[9629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:21:06.610958 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:06.985816 9653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:06.986131 9653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:06.986271 9653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:06.993721 9653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:06.993853 9653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:06.993935 9653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:06.994014 9653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:06.994071 9653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:07.004593 9653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:07.006071 9653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:07.007476 9653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:07.019397 9662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:07.019372 9665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:07.019501 9663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:07.020169 9653 server_base.cc:1034] running on GCE node
I20260811 12:21:07.020905 9653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:07.022378 9653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:07.023607 9653 hybrid_clock.cc:648] HybridClock initialized: now 1786450867023538 us; error 31 us; skew 500 ppm
I20260811 12:21:07.023947 9653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:07 dist-test-slave-rmhg krb5kdc[9629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450867, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:07.027079 9653 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:21:07.028512 9653 webserver.cc:460] Webserver started at http://127.0.0.1:44619/ using document root <none> and password file <none>
I20260811 12:21:07.029280 9653 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:07.029403 9653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:07.029737 9653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:07.032793 9653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "fe0822415545403f9270ca150483b6b8"
format_stamp: "Formatted at 2026-08-11 12:21:07 on dist-test-slave-rmhg"
server_key: "b3c1f7530105f04f6cef43b1860827fb"
server_key_iv: "9b8dd058de9f059775343b465ba63c0e"
server_key_version: "encryptionkey@0"
I20260811 12:21:07.033645 9653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "fe0822415545403f9270ca150483b6b8"
format_stamp: "Formatted at 2026-08-11 12:21:07 on dist-test-slave-rmhg"
server_key: "b3c1f7530105f04f6cef43b1860827fb"
server_key_iv: "9b8dd058de9f059775343b465ba63c0e"
server_key_version: "encryptionkey@0"
I20260811 12:21:07.039839 9653 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 12:21:07.043946 9678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:07.045508 9653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:21:07.045718 9653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "fe0822415545403f9270ca150483b6b8"
format_stamp: "Formatted at 2026-08-11 12:21:07 on dist-test-slave-rmhg"
server_key: "b3c1f7530105f04f6cef43b1860827fb"
server_key_iv: "9b8dd058de9f059775343b465ba63c0e"
server_key_version: "encryptionkey@0"
I20260811 12:21:07.045935 9653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:07.096294 9653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:07.100760 9653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:07.101063 9653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:07.120190 9653 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33051
I20260811 12:21:07.120213 9759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33051 every 8 connection(s)
I20260811 12:21:07.121748 9653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:07.125792 9760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:07.129010 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9653
I20260811 12:21:07.129240 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:07.129856 7324 external_mini_cluster.cc:1428] Setting key 99ebdd792b2fda6546c5699bac220dd1
I20260811 12:21:07.138586 9760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8: Bootstrap starting.
I20260811 12:21:07.142618 9760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:07.144073 9760 log.cc:824] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:21:07 dist-test-slave-rmhg krb5kdc[9629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:21:07.147580 9760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8: No bootstrap required, opened a new log
I20260811 12:21:07.155817 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:07.131971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:58879 (local address 127.0.0.1:33051)
0811 12:21:07.132829 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:07.132844 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:07.132889 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:21:07.134768 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:07.134801 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:07.135274 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:07.135693 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:07.137252 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:07.138616 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:07.140518 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:07.141319 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:07.141460 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:07.146623 (+ 5163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:07.146638 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:07.146661 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:07.146711 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:07.150767 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:07.151743 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:07.151752 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:07.151765 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:07.151989 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:07.152408 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:07.152412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:07.152416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:07.153191 (+ 775us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:07.153538 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:07.153944 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 12:21:07.154319 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":378,"threads_started":1}
I20260811 12:21:07.158306 9760 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0822415545403f9270ca150483b6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33051 } }
I20260811 12:21:07.158816 9760 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:07.158927 9760 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0822415545403f9270ca150483b6b8, State: Initialized, Role: FOLLOWER
I20260811 12:21:07.159603 9760 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0822415545403f9270ca150483b6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33051 } }
I20260811 12:21:07.159866 9760 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:07.160014 9760 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:07.160208 9760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:07.160239 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:07.162544 9760 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0822415545403f9270ca150483b6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33051 } }
I20260811 12:21:07.163178 9760 leader_election.cc:302] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe0822415545403f9270ca150483b6b8; no voters:
I20260811 12:21:07.163837 9760 leader_election.cc:288] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:07.163941 9773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:07.165079 9773 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [term 1 LEADER]: Becoming Leader. State: Replica: fe0822415545403f9270ca150483b6b8, State: Running, Role: LEADER
I20260811 12:21:07.165786 9773 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0822415545403f9270ca150483b6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33051 } }
I20260811 12:21:07.171352 9760 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:07.171883 9774 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe0822415545403f9270ca150483b6b8. Latest consensus state: current_term: 1 leader_uuid: "fe0822415545403f9270ca150483b6b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0822415545403f9270ca150483b6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33051 } } }
I20260811 12:21:07.172322 9774 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:07.172976 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe0822415545403f9270ca150483b6b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0822415545403f9270ca150483b6b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33051 } } }
I20260811 12:21:07.173342 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:07.175702 9779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:07.182284 9779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:07.191591 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:07.164995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:33761 (local address 127.0.0.1:33051)
0811 12:21:07.165245 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:07.165252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:07.170271 (+ 5019us) server_negotiation.cc:407] Connection header received
0811 12:21:07.170475 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:07.170480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:07.170581 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:07.170676 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:07.172949 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:07.174279 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:07.176213 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:07.176842 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:07.176962 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:07.180287 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:07.180294 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:07.180299 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:07.180346 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:07.184578 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:07.185996 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:07.186003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:07.186007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:07.186206 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:07.189411 (+ 3205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:07.189417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:07.189421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:07.189916 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:07.190182 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:07.190973 (+ 791us) server_negotiation.cc:299] Negotiation successful
0811 12:21:07.191208 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:21:07.195708 9779 catalog_manager.cc:1347] Generated new cluster ID: ef8f3a36bd38444a96d2b5d3b6d7ae7f
I20260811 12:21:07.195842 9779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:07.227383 9779 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:07.228747 9779 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:07.244489 9779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe0822415545403f9270ca150483b6b8: Generated new TSK 0
I20260811 12:21:07.245536 9779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:07.259209 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9653
I20260811 12:21:07.260427 9755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:07.356535 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9653
W20260811 12:21:07.381644 9626 connection.cc:535] client connection to 127.0.0.1:33051 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33051): Connection reset by peer (error 104)
2026-08-11T12:21:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:07 dist-test-slave-rmhg krb5kdc[9813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:07 dist-test-slave-rmhg krb5kdc[9813](info): set up 2 sockets
Aug 11 12:21:07 dist-test-slave-rmhg krb5kdc[9813](info): commencing operation
krb5kdc: starting...
W20260811 12:21:08.634793 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.003s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:08 dist-test-slave-rmhg krb5kdc[9813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:21:08.980681 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:09.346073 9833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:09.346410 9833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:09.346517 9833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:09.354223 9833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:09.354360 9833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:09.354408 9833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:09.354477 9833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:09.354529 9833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:09.365428 9833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36621
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:09.366856 9833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:09.368274 9833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:09.384383 9842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:09.384287 9845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:09.384262 9843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:09.385200 9833 server_base.cc:1034] running on GCE node
I20260811 12:21:09.386124 9833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:09.387703 9833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:09.388926 9833 hybrid_clock.cc:648] HybridClock initialized: now 1786450869388874 us; error 54 us; skew 500 ppm
I20260811 12:21:09.389344 9833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:09 dist-test-slave-rmhg krb5kdc[9813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450869, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:09.392686 9833 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:21:09.394379 9833 webserver.cc:460] Webserver started at http://127.0.0.1:36637/ using document root <none> and password file <none>
I20260811 12:21:09.395134 9833 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:09.395267 9833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:09.395584 9833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:09.399144 9833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "61a07e896b6d4bf8bf86823cee1e4726"
format_stamp: "Formatted at 2026-08-11 12:21:09 on dist-test-slave-rmhg"
server_key: "79add57a5bd12012b7862de1a1c0828f"
server_key_iv: "d918f06930d7e899dc8cf544def1ef9b"
server_key_version: "encryptionkey@0"
I20260811 12:21:09.400120 9833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "61a07e896b6d4bf8bf86823cee1e4726"
format_stamp: "Formatted at 2026-08-11 12:21:09 on dist-test-slave-rmhg"
server_key: "79add57a5bd12012b7862de1a1c0828f"
server_key_iv: "d918f06930d7e899dc8cf544def1ef9b"
server_key_version: "encryptionkey@0"
I20260811 12:21:09.406190 9833 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 12:21:09.410152 9858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:09.411680 9833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:21:09.411891 9833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "61a07e896b6d4bf8bf86823cee1e4726"
format_stamp: "Formatted at 2026-08-11 12:21:09 on dist-test-slave-rmhg"
server_key: "79add57a5bd12012b7862de1a1c0828f"
server_key_iv: "d918f06930d7e899dc8cf544def1ef9b"
server_key_version: "encryptionkey@0"
I20260811 12:21:09.412127 9833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:09.430051 9833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:09.434806 9833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:09.435127 9833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:09.454551 9833 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36621
I20260811 12:21:09.454581 9939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36621 every 8 connection(s)
I20260811 12:21:09.456072 9833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:09.460150 9941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:09.465438 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 9833
I20260811 12:21:09.465623 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:09.466058 7324 external_mini_cluster.cc:1428] Setting key 5387ff5071fb0a389dac07cb8beaa8a5
I20260811 12:21:09.472211 9941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726: Bootstrap starting.
I20260811 12:21:09.476156 9941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:09.477600 9941 log.cc:824] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:09.480847 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:09.468041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:33043 (local address 127.0.0.1:36621)
0811 12:21:09.468883 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:09.468897 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:09.468944 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:21:09.470819 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:09.470846 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:09.471287 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:09.471653 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:09.472375 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:09.473847 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:09.475717 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:09.476484 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:09.476659 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:09.478006 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:09.478015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:09.478031 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:09.478873 (+ 842us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:09.478987 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:09.479028 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:09.479208 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:09.479495 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 12:21:09.479746 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":357,"threads_started":1}
I20260811 12:21:09.481237 9941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726: No bootstrap required, opened a new log
I20260811 12:21:09.485054 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:09.491101 9941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a07e896b6d4bf8bf86823cee1e4726" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36621 } }
I20260811 12:21:09.491670 9941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:09.491802 9941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a07e896b6d4bf8bf86823cee1e4726, State: Initialized, Role: FOLLOWER
I20260811 12:21:09.492552 9941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a07e896b6d4bf8bf86823cee1e4726" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36621 } }
I20260811 12:21:09.492862 9941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:09.493010 9941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:09.493256 9941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:09.496213 9941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a07e896b6d4bf8bf86823cee1e4726" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36621 } }
I20260811 12:21:09.496942 9941 leader_election.cc:302] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61a07e896b6d4bf8bf86823cee1e4726; no voters:
I20260811 12:21:09.497574 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:09.490605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:37341 (local address 127.0.0.1:36621)
0811 12:21:09.490844 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:09.490850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:09.491226 (+ 376us) server_negotiation.cc:407] Connection header received
0811 12:21:09.491368 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:09.491374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:09.491462 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:09.491594 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:09.492129 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:09.493320 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:09.494776 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:09.495497 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:09.495629 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:09.496764 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:09.496771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:09.496775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:09.496800 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:09.496874 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:09.496896 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:09.497027 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:09.497175 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 12:21:09.497239 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:21:09.497792 9941 leader_election.cc:288] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:09.497933 9955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:09.498629 9955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [term 1 LEADER]: Becoming Leader. State: Replica: 61a07e896b6d4bf8bf86823cee1e4726, State: Running, Role: LEADER
I20260811 12:21:09.499332 9955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a07e896b6d4bf8bf86823cee1e4726" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36621 } }
I20260811 12:21:09.501533 9955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61a07e896b6d4bf8bf86823cee1e4726. Latest consensus state: current_term: 1 leader_uuid: "61a07e896b6d4bf8bf86823cee1e4726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a07e896b6d4bf8bf86823cee1e4726" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36621 } } }
I20260811 12:21:09.501904 9955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:09.502506 9957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61a07e896b6d4bf8bf86823cee1e4726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a07e896b6d4bf8bf86823cee1e4726" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36621 } } }
I20260811 12:21:09.502912 9957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:09.503363 9941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:09.508817 9959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:09.513974 9959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:09.524353 9959 catalog_manager.cc:1347] Generated new cluster ID: 160f3f85350f4af7a8a810cd28458f3f
I20260811 12:21:09.524506 9959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:09.538398 9959 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:09.539790 9959 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:09.554359 9959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61a07e896b6d4bf8bf86823cee1e4726: Generated new TSK 0
I20260811 12:21:09.555352 9959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:09.566471 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9833
W20260811 12:21:09.568262 9932 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.204:33043
W20260811 12:21:09.569289 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 9833: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:21:09.569469 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 9833
W20260811 12:21:09.599776 9807 connection.cc:535] client connection to 127.0.0.1:36621 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36621): Connection reset by peer (error 104)
2026-08-11T12:21:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2217 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:09 dist-test-slave-rmhg krb5kdc[9987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:09 dist-test-slave-rmhg krb5kdc[9987](info): set up 2 sockets
Aug 11 12:21:09 dist-test-slave-rmhg krb5kdc[9987](info): commencing operation
krb5kdc: starting...
W20260811 12:21:10.844902 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:11 dist-test-slave-rmhg krb5kdc[9987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:21:11.197103 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:11.565328 10008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:11.565694 10008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:11.565842 10008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:11.573725 10008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:11.573890 10008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:11.573977 10008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:11.574062 10008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:11.574131 10008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:11.584681 10008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32919
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:11.586226 10008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:11.587642 10008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:11.600286 10018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:11.600322 10019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:11.600344 10021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:11.601380 10008 server_base.cc:1034] running on GCE node
I20260811 12:21:11.602267 10008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:11.603818 10008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:11.605070 10008 hybrid_clock.cc:648] HybridClock initialized: now 1786450871605001 us; error 50 us; skew 500 ppm
I20260811 12:21:11.605504 10008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:11 dist-test-slave-rmhg krb5kdc[9987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:11.608865 10008 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:21:11.610530 10008 webserver.cc:460] Webserver started at http://127.0.0.1:33521/ using document root <none> and password file <none>
I20260811 12:21:11.611270 10008 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:11.611390 10008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:11.611707 10008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:11.615072 10008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "7cb2072f58884bc2a6ce1b8513d9298d"
format_stamp: "Formatted at 2026-08-11 12:21:11 on dist-test-slave-rmhg"
server_key: "900d9c9d987816407b67174db7324d47"
server_key_iv: "77c8cc43a6835715d76f8262319762a4"
server_key_version: "encryptionkey@0"
I20260811 12:21:11.615907 10008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "7cb2072f58884bc2a6ce1b8513d9298d"
format_stamp: "Formatted at 2026-08-11 12:21:11 on dist-test-slave-rmhg"
server_key: "900d9c9d987816407b67174db7324d47"
server_key_iv: "77c8cc43a6835715d76f8262319762a4"
server_key_version: "encryptionkey@0"
I20260811 12:21:11.621624 10008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 12:21:11.625422 10035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:11.626861 10008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:21:11.627048 10008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "7cb2072f58884bc2a6ce1b8513d9298d"
format_stamp: "Formatted at 2026-08-11 12:21:11 on dist-test-slave-rmhg"
server_key: "900d9c9d987816407b67174db7324d47"
server_key_iv: "77c8cc43a6835715d76f8262319762a4"
server_key_version: "encryptionkey@0"
I20260811 12:21:11.627246 10008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:11.633424 10008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:11.638010 10008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:11.638334 10008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:11.657330 10008 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32919
I20260811 12:21:11.657353 10125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32919 every 8 connection(s)
I20260811 12:21:11.658865 10008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:11.662451 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10008
I20260811 12:21:11.662627 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:11.662987 10127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:11.663080 7324 external_mini_cluster.cc:1428] Setting key ba27b6b7b2523c6a514d3d679d18676d
I20260811 12:21:11.672462 10132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:11.664999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:39685 (local address 127.0.0.1:32919)
0811 12:21:11.665865 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:11.665881 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:11.665943 (+ 62us) server_negotiation.cc:407] Connection header received
0811 12:21:11.668227 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:11.668260 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:11.668762 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:11.669283 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:11.670353 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:11.670362 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:11.670380 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:11.670398 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:11.670485 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:11.670512 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:11.670757 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:11.670905 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 12:21:11.671143 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":334,"threads_started":1}
I20260811 12:21:11.676762 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:11.678493 10127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d: Bootstrap starting.
I20260811 12:21:11.682317 10127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:11.683622 10127 log.cc:824] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:11.686688 10127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d: No bootstrap required, opened a new log
I20260811 12:21:11.687625 10132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:11.681401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.204:55655 (local address 127.0.0.1:32919)
0811 12:21:11.681592 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:11.681599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:11.682017 (+ 418us) server_negotiation.cc:407] Connection header received
0811 12:21:11.682208 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:11.682216 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:11.682271 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:11.682364 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:11.686926 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:11.686931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:11.686934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:11.686952 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:11.686995 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:11.687010 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:11.687112 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:11.687296 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 12:21:11.687358 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:21:11.695222 10127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb2072f58884bc2a6ce1b8513d9298d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32919 } }
I20260811 12:21:11.695649 10127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:11.695751 10127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb2072f58884bc2a6ce1b8513d9298d, State: Initialized, Role: FOLLOWER
I20260811 12:21:11.696343 10127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb2072f58884bc2a6ce1b8513d9298d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32919 } }
I20260811 12:21:11.696590 10127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:11.696725 10127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:11.696888 10127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:11.699613 10127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb2072f58884bc2a6ce1b8513d9298d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32919 } }
I20260811 12:21:11.700158 10127 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cb2072f58884bc2a6ce1b8513d9298d; no voters:
I20260811 12:21:11.700769 10127 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:11.700917 10142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:11.701613 10142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [term 1 LEADER]: Becoming Leader. State: Replica: 7cb2072f58884bc2a6ce1b8513d9298d, State: Running, Role: LEADER
I20260811 12:21:11.702152 10142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb2072f58884bc2a6ce1b8513d9298d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32919 } }
I20260811 12:21:11.702926 10127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:11.704847 10143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cb2072f58884bc2a6ce1b8513d9298d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb2072f58884bc2a6ce1b8513d9298d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32919 } } }
I20260811 12:21:11.705220 10143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:11.706097 10151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:11.705911 10145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb2072f58884bc2a6ce1b8513d9298d. Latest consensus state: current_term: 1 leader_uuid: "7cb2072f58884bc2a6ce1b8513d9298d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb2072f58884bc2a6ce1b8513d9298d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32919 } } }
I20260811 12:21:11.708338 10145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:11.712119 10151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:11.723033 10151 catalog_manager.cc:1347] Generated new cluster ID: 8db626e15d564a969db481a1e38722bd
I20260811 12:21:11.723189 10151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:11.748862 10151 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:11.749475 10151 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:11.759125 10151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cb2072f58884bc2a6ce1b8513d9298d: Generated new TSK 0
I20260811 12:21:11.760352 10151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:11.775633 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10008
W20260811 12:21:11.776901 10116 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.204:39685
W20260811 12:21:11.777737 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10008: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:21:11.777917 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10008
2026-08-11T12:21:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2204 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:11 dist-test-slave-rmhg krb5kdc[10171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:11 dist-test-slave-rmhg krb5kdc[10171](info): set up 2 sockets
Aug 11 12:21:11 dist-test-slave-rmhg krb5kdc[10171](info): commencing operation
krb5kdc: starting...
W20260811 12:21:13.053081 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.013s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:13 dist-test-slave-rmhg krb5kdc[10171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:21:13.413472 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35197
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:33069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:35197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:13.783780 10192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:13.784118 10192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:13.784212 10192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:13.791793 10192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:13.791935 10192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:13.791996 10192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:13.792057 10192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:13.792101 10192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:13.802503 10192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:33069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:35197
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:13.803925 10192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:13.805397 10192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:13.817751 10198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:13.817938 10199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:13.817951 10201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:13.819152 10192 server_base.cc:1034] running on GCE node
I20260811 12:21:13.820006 10192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:13.821679 10192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:13.822904 10192 hybrid_clock.cc:648] HybridClock initialized: now 1786450873822860 us; error 36 us; skew 500 ppm
I20260811 12:21:13.823294 10192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:13 dist-test-slave-rmhg krb5kdc[10171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450873, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:13.826661 10192 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:21:13.828236 10192 webserver.cc:460] Webserver started at http://127.7.39.62:44109/ using document root <none> and password file <none>
I20260811 12:21:13.828945 10192 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:13.829052 10192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:13.829383 10192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:13.832556 10192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "44a97dc7c3614763938730786a11b37e"
format_stamp: "Formatted at 2026-08-11 12:21:13 on dist-test-slave-rmhg"
server_key: "3294ca8d5eed67ff178c134aeaf298f2"
server_key_iv: "9df296c3c82f93e106259b35847647dd"
server_key_version: "encryptionkey@0"
I20260811 12:21:13.833397 10192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "44a97dc7c3614763938730786a11b37e"
format_stamp: "Formatted at 2026-08-11 12:21:13 on dist-test-slave-rmhg"
server_key: "3294ca8d5eed67ff178c134aeaf298f2"
server_key_iv: "9df296c3c82f93e106259b35847647dd"
server_key_version: "encryptionkey@0"
I20260811 12:21:13.839156 10192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:21:13.843262 10210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:13.844808 10192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:21:13.845005 10192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "44a97dc7c3614763938730786a11b37e"
format_stamp: "Formatted at 2026-08-11 12:21:13 on dist-test-slave-rmhg"
server_key: "3294ca8d5eed67ff178c134aeaf298f2"
server_key_iv: "9df296c3c82f93e106259b35847647dd"
server_key_version: "encryptionkey@0"
I20260811 12:21:13.845265 10192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:13.869879 10192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:13.874431 10192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:13.874728 10192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:13.894048 10192 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:35197
I20260811 12:21:13.894085 10289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:35197 every 8 connection(s)
I20260811 12:21:13.895509 10192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:13.899623 10290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:13.900513 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10192
I20260811 12:21:13.900677 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:13.901115 7324 external_mini_cluster.cc:1428] Setting key 18bee0a774c74dd53da63960c0d8b2d8
I20260811 12:21:13.914413 10290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e: Bootstrap starting.
Aug 11 12:21:13 dist-test-slave-rmhg krb5kdc[10171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:13.918352 10290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:13.919620 10290 log.cc:824] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:13.923427 10290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e: No bootstrap required, opened a new log
I20260811 12:21:13.926879 10293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:13.903200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45099 (local address 127.7.39.62:35197)
0811 12:21:13.904118 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:13.904134 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:13.904192 (+ 58us) server_negotiation.cc:407] Connection header received
0811 12:21:13.906083 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:13.906111 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:13.906596 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:13.906966 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:13.908826 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:13.910276 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:13.912190 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:13.912920 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:13.913055 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:13.918156 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:13.918167 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:13.918185 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:13.918234 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:13.921701 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:13.922916 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:13.922927 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:13.922937 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:13.923206 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:13.923762 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:13.923768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:13.923772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:13.924378 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:13.924733 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:13.925369 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 12:21:13.925721 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":416,"threads_started":1}
I20260811 12:21:13.931146 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:13.932216 10290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a97dc7c3614763938730786a11b37e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35197 } }
I20260811 12:21:13.932646 10290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:13.932746 10290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a97dc7c3614763938730786a11b37e, State: Initialized, Role: FOLLOWER
I20260811 12:21:13.933418 10290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a97dc7c3614763938730786a11b37e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35197 } }
I20260811 12:21:13.933673 10290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:13.933815 10290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:13.934015 10290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:13.936866 10290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a97dc7c3614763938730786a11b37e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35197 } }
I20260811 12:21:13.937583 10290 leader_election.cc:302] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44a97dc7c3614763938730786a11b37e; no voters:
I20260811 12:21:13.938297 10290 leader_election.cc:288] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:13.938392 10302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:13.939057 10302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [term 1 LEADER]: Becoming Leader. State: Replica: 44a97dc7c3614763938730786a11b37e, State: Running, Role: LEADER
I20260811 12:21:13.939815 10302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a97dc7c3614763938730786a11b37e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35197 } }
I20260811 12:21:13.942229 10290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:13.942723 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44a97dc7c3614763938730786a11b37e. Latest consensus state: current_term: 1 leader_uuid: "44a97dc7c3614763938730786a11b37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a97dc7c3614763938730786a11b37e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35197 } } }
I20260811 12:21:13.943132 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:13.943727 10304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44a97dc7c3614763938730786a11b37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a97dc7c3614763938730786a11b37e" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35197 } } }
I20260811 12:21:13.944058 10304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:13.955902 10293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:13.935619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56131 (local address 127.7.39.62:35197)
0811 12:21:13.935831 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:13.935838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:13.936314 (+ 476us) server_negotiation.cc:407] Connection header received
0811 12:21:13.936476 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:13.936480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:13.936544 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:13.936624 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:13.938690 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:13.939946 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:13.941877 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:13.942568 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:13.942694 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:13.946419 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:13.946427 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:13.946433 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:13.946484 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:13.950967 (+ 4483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:13.952247 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:13.952254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:13.952259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:13.952436 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:13.953043 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:13.953048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:13.953052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:13.953580 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:13.953888 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:13.955269 (+ 1381us) server_negotiation.cc:299] Negotiation successful
0811 12:21:13.955486 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:21:13.967099 10307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:13.973574 10307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:13.985230 10307 catalog_manager.cc:1347] Generated new cluster ID: 593d424d76674218bb11132daee151e3
I20260811 12:21:13.985374 10307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:14.014189 10307 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:14.016182 10307 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:14.048367 10307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44a97dc7c3614763938730786a11b37e: Generated new TSK 0
I20260811 12:21:14.049633 10307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:14.065711 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10192
I20260811 12:21:14.067449 10284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:14.166062 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10192
W20260811 12:21:14.189620 10169 connection.cc:535] client connection to 127.7.39.62:35197 recv error: Network error: failed to read from TLS socket (remote: 127.7.39.62:35197): Connection reset by peer (error 104)
2026-08-11T12:21:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:14 dist-test-slave-rmhg krb5kdc[10338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:14 dist-test-slave-rmhg krb5kdc[10338](info): set up 2 sockets
Aug 11 12:21:14 dist-test-slave-rmhg krb5kdc[10338](info): commencing operation
krb5kdc: starting...
W20260811 12:21:15.440272 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:15 dist-test-slave-rmhg krb5kdc[10338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:21:15.788808 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:46155
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:33423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:46155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:16.154402 10356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:16.154759 10356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:16.154908 10356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:16.162607 10356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:16.162775 10356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:16.162864 10356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:16.162951 10356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:16.163015 10356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:16.173858 10356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:33423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:46155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:46155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:16.175344 10356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:16.176847 10356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:16.189494 10363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:16.189623 10361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:16.189599 10365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:16.190196 10356 server_base.cc:1034] running on GCE node
I20260811 12:21:16.191144 10356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:16.192821 10356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:16.194077 10356 hybrid_clock.cc:648] HybridClock initialized: now 1786450876193979 us; error 79 us; skew 500 ppm
I20260811 12:21:16.194495 10356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:16 dist-test-slave-rmhg krb5kdc[10338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450876, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:16.198010 10356 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:21:16.199769 10356 webserver.cc:460] Webserver started at http://127.7.39.62:45083/ using document root <none> and password file <none>
I20260811 12:21:16.200605 10356 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:16.200762 10356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:16.201121 10356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:16.204614 10356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "2ec2c31e4a494742a976255d5109bff5"
format_stamp: "Formatted at 2026-08-11 12:21:16 on dist-test-slave-rmhg"
server_key: "82f9eadf23765c4100eaf5daa1dba768"
server_key_iv: "2da5b1e4f084a1e7893af26b2f9fd382"
server_key_version: "encryptionkey@0"
I20260811 12:21:16.205520 10356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "2ec2c31e4a494742a976255d5109bff5"
format_stamp: "Formatted at 2026-08-11 12:21:16 on dist-test-slave-rmhg"
server_key: "82f9eadf23765c4100eaf5daa1dba768"
server_key_iv: "2da5b1e4f084a1e7893af26b2f9fd382"
server_key_version: "encryptionkey@0"
I20260811 12:21:16.211398 10356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 12:21:16.215476 10379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:16.217023 10356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 12:21:16.217252 10356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "2ec2c31e4a494742a976255d5109bff5"
format_stamp: "Formatted at 2026-08-11 12:21:16 on dist-test-slave-rmhg"
server_key: "82f9eadf23765c4100eaf5daa1dba768"
server_key_iv: "2da5b1e4f084a1e7893af26b2f9fd382"
server_key_version: "encryptionkey@0"
I20260811 12:21:16.217478 10356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:16.246461 10356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:16.251389 10356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:16.251757 10356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:16.269662 10356 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:46155
I20260811 12:21:16.269704 10461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:46155 every 8 connection(s)
I20260811 12:21:16.271092 10356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:16.274855 10463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:16.275161 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10356
I20260811 12:21:16.275332 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:16.275760 7324 external_mini_cluster.cc:1428] Setting key a8d3c0f5095c766b2ac0dff08bf18d42
I20260811 12:21:16.290385 10463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5: Bootstrap starting.
I20260811 12:21:16.292918 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:16.277671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33661 (local address 127.7.39.62:46155)
0811 12:21:16.278571 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:16.278587 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:16.278647 (+ 60us) server_negotiation.cc:407] Connection header received
0811 12:21:16.281609 (+ 2962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:16.281651 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:16.282374 (+ 723us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:16.282936 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:16.283511 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:16.285498 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:16.287009 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:16.288179 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:16.288362 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:16.289831 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:16.289843 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:16.289865 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:16.290654 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:16.290769 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:16.290806 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:16.290986 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:16.291273 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 12:21:16.291468 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":374,"threads_started":1}
I20260811 12:21:16.294626 10463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:16.295995 10463 log.cc:824] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:16.297782 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:16.299718 10463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5: No bootstrap required, opened a new log
I20260811 12:21:16.310216 10463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec2c31e4a494742a976255d5109bff5" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46155 } }
I20260811 12:21:16.310635 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:16.303236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43087 (local address 127.7.39.62:46155)
0811 12:21:16.303444 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:16.303450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:16.303795 (+ 345us) server_negotiation.cc:407] Connection header received
0811 12:21:16.304028 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:16.304033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:16.304107 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:16.304209 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:16.304865 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:16.305770 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:16.307751 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:16.308270 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:16.308379 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:16.309775 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:16.309780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:16.309784 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:16.309805 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:16.309867 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:16.309883 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:16.310000 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:16.310256 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 12:21:16.310315 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:21:16.310765 10463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:16.310879 10463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec2c31e4a494742a976255d5109bff5, State: Initialized, Role: FOLLOWER
I20260811 12:21:16.311553 10463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec2c31e4a494742a976255d5109bff5" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46155 } }
I20260811 12:21:16.311841 10463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:16.311985 10463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:16.312157 10463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:16.314525 10463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec2c31e4a494742a976255d5109bff5" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46155 } }
I20260811 12:21:16.315070 10463 leader_election.cc:302] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ec2c31e4a494742a976255d5109bff5; no voters:
I20260811 12:21:16.315692 10463 leader_election.cc:288] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:16.315832 10480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:16.316574 10480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec2c31e4a494742a976255d5109bff5, State: Running, Role: LEADER
I20260811 12:21:16.317214 10480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec2c31e4a494742a976255d5109bff5" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46155 } }
I20260811 12:21:16.317948 10463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:16.323773 10483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec2c31e4a494742a976255d5109bff5. Latest consensus state: current_term: 1 leader_uuid: "2ec2c31e4a494742a976255d5109bff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec2c31e4a494742a976255d5109bff5" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46155 } } }
I20260811 12:21:16.324268 10483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:16.324895 10481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ec2c31e4a494742a976255d5109bff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec2c31e4a494742a976255d5109bff5" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 46155 } } }
I20260811 12:21:16.325246 10481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:16.325523 10503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:16.331712 10503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:16.341305 10503 catalog_manager.cc:1347] Generated new cluster ID: b0b169d3ebfe4894a553d718adf77b18
I20260811 12:21:16.341445 10503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:16.365254 10503 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:16.367316 10503 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:16.385812 10503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ec2c31e4a494742a976255d5109bff5: Generated new TSK 0
I20260811 12:21:16.386785 10503 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:16.404181 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10356
W20260811 12:21:16.405577 10456 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33661
W20260811 12:21:16.406508 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10356: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:21:16.406679 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10356
W20260811 12:21:16.432567 10336 connection.cc:535] client connection to 127.7.39.62:46155 recv error: Network error: failed to read from TLS socket (remote: 127.7.39.62:46155): Connection reset by peer (error 104)
2026-08-11T12:21:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:16 dist-test-slave-rmhg krb5kdc[10520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:16 dist-test-slave-rmhg krb5kdc[10520](info): set up 2 sockets
Aug 11 12:21:16 dist-test-slave-rmhg krb5kdc[10520](info): commencing operation
krb5kdc: starting...
W20260811 12:21:17.690222 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:17 dist-test-slave-rmhg krb5kdc[10520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:21:18.040314 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34487
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:44873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:34487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:18.404942 10540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:18.405385 10540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:18.405583 10540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:18.413343 10540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:18.413492 10540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:18.413585 10540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:18.413678 10540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:18.413743 10540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:18.424245 10540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:44873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:34487
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:34487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:18.425788 10540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:18.427253 10540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:18.440073 10553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:18.440033 10559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:18.440021 10556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:18.440927 10540 server_base.cc:1034] running on GCE node
I20260811 12:21:18.441936 10540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:18.443735 10540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:18.444970 10540 hybrid_clock.cc:648] HybridClock initialized: now 1786450878444910 us; error 69 us; skew 500 ppm
I20260811 12:21:18.445417 10540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:18 dist-test-slave-rmhg krb5kdc[10520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:18.449007 10540 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:21:18.450740 10540 webserver.cc:460] Webserver started at http://127.7.39.62:40117/ using document root <none> and password file <none>
I20260811 12:21:18.451543 10540 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:18.451681 10540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:18.452023 10540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:18.455411 10540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6"
format_stamp: "Formatted at 2026-08-11 12:21:18 on dist-test-slave-rmhg"
server_key: "63164a311927aa86d62d2fa5a268faf7"
server_key_iv: "f2bed663a974237b331b92b0d2469d54"
server_key_version: "encryptionkey@0"
I20260811 12:21:18.456256 10540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6"
format_stamp: "Formatted at 2026-08-11 12:21:18 on dist-test-slave-rmhg"
server_key: "63164a311927aa86d62d2fa5a268faf7"
server_key_iv: "f2bed663a974237b331b92b0d2469d54"
server_key_version: "encryptionkey@0"
I20260811 12:21:18.462090 10540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:21:18.465988 10570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:18.467594 10540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:21:18.467860 10540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6"
format_stamp: "Formatted at 2026-08-11 12:21:18 on dist-test-slave-rmhg"
server_key: "63164a311927aa86d62d2fa5a268faf7"
server_key_iv: "f2bed663a974237b331b92b0d2469d54"
server_key_version: "encryptionkey@0"
I20260811 12:21:18.468115 10540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:18.474301 10540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:18.478885 10540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:18.479197 10540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:18.497731 10540 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:34487
I20260811 12:21:18.497784 10665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:34487 every 8 connection(s)
I20260811 12:21:18.499243 10540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:18.503276 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:18.505930 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10540
I20260811 12:21:18.506089 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:18.506604 7324 external_mini_cluster.cc:1428] Setting key 493c601b330d80acfc07058f8842d0dd
I20260811 12:21:18.514871 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:18.508486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49883 (local address 127.7.39.62:34487)
0811 12:21:18.509379 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:18.509392 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:18.509438 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:21:18.511156 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:18.511181 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:18.511554 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:18.511910 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:18.513106 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:18.513113 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:18.513127 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:18.513147 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:18.513277 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:18.513302 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:18.513572 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:18.513698 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 12:21:18.513909 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":380,"threads_started":1}
I20260811 12:21:18.515712 10668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6: Bootstrap starting.
I20260811 12:21:18.519088 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:18.519368 10668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:18.520421 10668 log.cc:824] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:18.523669 10668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6: No bootstrap required, opened a new log
I20260811 12:21:18.526466 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:18.523584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33173 (local address 127.7.39.62:34487)
0811 12:21:18.523777 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:18.523784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:18.524232 (+ 448us) server_negotiation.cc:407] Connection header received
0811 12:21:18.524380 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:18.524385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:18.524466 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:18.524590 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:18.525710 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:18.525716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:18.525721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:18.525741 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:18.525799 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:18.525819 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:18.525951 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:18.526104 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 12:21:18.526163 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:21:18.532414 10668 raft_consensus.cc:348] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34487 } }
I20260811 12:21:18.532869 10668 raft_consensus.cc:374] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:18.532961 10668 raft_consensus.cc:729] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48048f00e3a4bb5ae70f7ae2ff0a3d6, State: Initialized, Role: FOLLOWER
I20260811 12:21:18.533568 10668 consensus_queue.cc:260] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34487 } }
I20260811 12:21:18.533795 10668 raft_consensus.cc:388] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:18.533897 10668 raft_consensus.cc:482] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:18.534024 10668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:18.536554 10668 raft_consensus.cc:504] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34487 } }
I20260811 12:21:18.537310 10668 leader_election.cc:302] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e48048f00e3a4bb5ae70f7ae2ff0a3d6; no voters:
I20260811 12:21:18.538025 10668 leader_election.cc:288] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:18.538138 10682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:18.538856 10682 raft_consensus.cc:686] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [term 1 LEADER]: Becoming Leader. State: Replica: e48048f00e3a4bb5ae70f7ae2ff0a3d6, State: Running, Role: LEADER
I20260811 12:21:18.539408 10682 consensus_queue.cc:237] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34487 } }
I20260811 12:21:18.540239 10668 sys_catalog.cc:564] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:18.543327 10685 sys_catalog.cc:455] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e48048f00e3a4bb5ae70f7ae2ff0a3d6. Latest consensus state: current_term: 1 leader_uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34487 } } }
I20260811 12:21:18.544238 10685 sys_catalog.cc:458] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:18.543928 10684 sys_catalog.cc:455] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48048f00e3a4bb5ae70f7ae2ff0a3d6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 34487 } } }
I20260811 12:21:18.544638 10684 sys_catalog.cc:458] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:18.545315 10696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:18.549913 10696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:18.558844 10696 catalog_manager.cc:1347] Generated new cluster ID: ce20644a8d434f368b1695947dc3810a
I20260811 12:21:18.558976 10696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:18.578553 10696 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:18.579026 10696 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:18.586604 10696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e48048f00e3a4bb5ae70f7ae2ff0a3d6: Generated new TSK 0
I20260811 12:21:18.587853 10696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:18.592034 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10540
W20260811 12:21:18.593531 10661 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49883
W20260811 12:21:18.594252 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10540: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:21:18.594424 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10540
2026-08-11T12:21:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2189 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:18 dist-test-slave-rmhg krb5kdc[10719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:18 dist-test-slave-rmhg krb5kdc[10719](info): set up 2 sockets
Aug 11 12:21:18 dist-test-slave-rmhg krb5kdc[10719](info): commencing operation
krb5kdc: starting...
W20260811 12:21:19.866755 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:20 dist-test-slave-rmhg krb5kdc[10719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:21:20.220482 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:33351
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:33351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:20.590291 10734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:20.590646 10734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:20.590776 10734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:20.598809 10734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:20.598954 10734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:20.599017 10734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:20.599103 10734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:20.599161 10734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:20.610013 10734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:33351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:33351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:20.611505 10734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:20.613006 10734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:20.625790 10743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:20.625733 10744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:20.625753 10746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:20.626693 10734 server_base.cc:1034] running on GCE node
I20260811 12:21:20.627629 10734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:20.629328 10734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:20.630575 10734 hybrid_clock.cc:648] HybridClock initialized: now 1786450880630516 us; error 60 us; skew 500 ppm
I20260811 12:21:20.630991 10734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:20 dist-test-slave-rmhg krb5kdc[10719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450880, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:20.634390 10734 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:21:20.636107 10734 webserver.cc:460] Webserver started at http://127.7.39.62:44589/ using document root <none> and password file <none>
I20260811 12:21:20.636876 10734 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:20.637017 10734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:20.637387 10734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:20.640828 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "86ddb6edddab472e8e91da336cc75631"
format_stamp: "Formatted at 2026-08-11 12:21:20 on dist-test-slave-rmhg"
server_key: "bd1a229632527918f9334d05a3736fb6"
server_key_iv: "3cb1e4db74104c19f056b8f526fef440"
server_key_version: "encryptionkey@0"
I20260811 12:21:20.641793 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "86ddb6edddab472e8e91da336cc75631"
format_stamp: "Formatted at 2026-08-11 12:21:20 on dist-test-slave-rmhg"
server_key: "bd1a229632527918f9334d05a3736fb6"
server_key_iv: "3cb1e4db74104c19f056b8f526fef440"
server_key_version: "encryptionkey@0"
I20260811 12:21:20.648162 10734 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 12:21:20.652330 10753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:20.653898 10734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:21:20.654093 10734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "86ddb6edddab472e8e91da336cc75631"
format_stamp: "Formatted at 2026-08-11 12:21:20 on dist-test-slave-rmhg"
server_key: "bd1a229632527918f9334d05a3736fb6"
server_key_iv: "3cb1e4db74104c19f056b8f526fef440"
server_key_version: "encryptionkey@0"
I20260811 12:21:20.654294 10734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:20.668010 10734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:20.672443 10734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:20.672727 10734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:20.691609 10734 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:33351
I20260811 12:21:20.691635 10836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:33351 every 8 connection(s)
I20260811 12:21:20.693197 10734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:20.696574 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10734
I20260811 12:21:20.696759 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:20.697264 7324 external_mini_cluster.cc:1428] Setting key 973008bc18785332d319672f8959459c
I20260811 12:21:20.697511 10838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:20.712610 10838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631: Bootstrap starting.
Aug 11 12:21:20 dist-test-slave-rmhg krb5kdc[10719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:20.716692 10838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:20.718242 10838 log.cc:824] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:20.722165 10838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631: No bootstrap required, opened a new log
I20260811 12:21:20.723884 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:20.699276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44187 (local address 127.7.39.62:33351)
0811 12:21:20.700158 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:20.700175 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:20.700230 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:21:20.702668 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:20.702705 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:20.703342 (+ 637us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:20.703883 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:20.705428 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:20.707409 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:20.708849 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:20.710053 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:20.710263 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:20.714534 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:20.714547 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:20.714570 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:20.714640 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:20.718651 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:20.719595 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:20.719602 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:20.719617 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:20.719805 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:20.720200 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:20.720205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:20.720208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:20.720952 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:20.721296 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:20.721816 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 12:21:20.722321 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":378,"threads_started":1}
I20260811 12:21:20.728538 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:20.731449 10838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ddb6edddab472e8e91da336cc75631" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33351 } }
I20260811 12:21:20.731887 10838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:20.732043 10838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ddb6edddab472e8e91da336cc75631, State: Initialized, Role: FOLLOWER
I20260811 12:21:20.732725 10838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ddb6edddab472e8e91da336cc75631" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33351 } }
I20260811 12:21:20.733023 10838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:20.733290 10838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:20.733551 10838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:20.736737 10838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ddb6edddab472e8e91da336cc75631" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33351 } }
I20260811 12:21:20.737412 10838 leader_election.cc:302] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86ddb6edddab472e8e91da336cc75631; no voters:
I20260811 12:21:20.738174 10838 leader_election.cc:288] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:20.738277 10855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:20.739557 10855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [term 1 LEADER]: Becoming Leader. State: Replica: 86ddb6edddab472e8e91da336cc75631, State: Running, Role: LEADER
I20260811 12:21:20.740357 10855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ddb6edddab472e8e91da336cc75631" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33351 } }
I20260811 12:21:20.740583 10838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:20.743731 10857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86ddb6edddab472e8e91da336cc75631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ddb6edddab472e8e91da336cc75631" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33351 } } }
I20260811 12:21:20.744184 10857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:20.746454 10858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86ddb6edddab472e8e91da336cc75631. Latest consensus state: current_term: 1 leader_uuid: "86ddb6edddab472e8e91da336cc75631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ddb6edddab472e8e91da336cc75631" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 33351 } } }
I20260811 12:21:20.747330 10858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:20.751509 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:20.733128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53421 (local address 127.7.39.62:33351)
0811 12:21:20.733368 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:20.733376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:20.733837 (+ 461us) server_negotiation.cc:407] Connection header received
0811 12:21:20.734032 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:20.734036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:20.734108 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:20.734198 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:20.736164 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:20.736958 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:20.738864 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:20.739530 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:20.739668 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:20.743151 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:20.743159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:20.743164 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:20.743216 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:20.747320 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:20.748436 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:20.748443 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:20.748447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:20.748642 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:20.749203 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:20.749209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:20.749213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:20.749711 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:20.750007 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:20.750726 (+ 719us) server_negotiation.cc:299] Negotiation successful
0811 12:21:20.751080 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:21:20.754740 10862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:20.759850 10862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:20.771178 10862 catalog_manager.cc:1347] Generated new cluster ID: d545f72e994c417aa615eea4d3197864
I20260811 12:21:20.771330 10862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:20.788350 10862 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:20.789918 10862 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:20.803490 10862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86ddb6edddab472e8e91da336cc75631: Generated new TSK 0
I20260811 12:21:20.804802 10862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:20.817253 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10734
I20260811 12:21:20.818663 10831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:20.917647 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10734
2026-08-11T12:21:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:21 dist-test-slave-rmhg krb5kdc[10882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:21 dist-test-slave-rmhg krb5kdc[10882](info): set up 2 sockets
Aug 11 12:21:21 dist-test-slave-rmhg krb5kdc[10882](info): commencing operation
krb5kdc: starting...
W20260811 12:21:22.206004 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:22 dist-test-slave-rmhg krb5kdc[10882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:21:22.564843 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:38921
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:38921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:22.933283 10901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:22.933609 10901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:22.933732 10901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:22.941401 10901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:22.941536 10901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:22.941623 10901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:22.941716 10901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:22.941773 10901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:22.952355 10901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:38921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:38921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:22.953821 10901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:22.955287 10901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:22.968009 10911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:22.967931 10915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:22.967980 10912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:22.968852 10901 server_base.cc:1034] running on GCE node
I20260811 12:21:22.969794 10901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:22.971396 10901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:22.972658 10901 hybrid_clock.cc:648] HybridClock initialized: now 1786450882972592 us; error 59 us; skew 500 ppm
I20260811 12:21:22.973107 10901 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:22 dist-test-slave-rmhg krb5kdc[10882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450882, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:22.984763 10901 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:21:22.986737 10901 webserver.cc:460] Webserver started at http://127.7.39.62:38391/ using document root <none> and password file <none>
I20260811 12:21:22.987543 10901 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:22.987690 10901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:22.988039 10901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:22.991480 10901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "25cfe566171b411aa65d169c7a3b1f9f"
format_stamp: "Formatted at 2026-08-11 12:21:22 on dist-test-slave-rmhg"
server_key: "16d786796fdb7a1c0d3144dd8c38f4c1"
server_key_iv: "56b063123a614591933aee636942ea7b"
server_key_version: "encryptionkey@0"
I20260811 12:21:22.992340 10901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "25cfe566171b411aa65d169c7a3b1f9f"
format_stamp: "Formatted at 2026-08-11 12:21:22 on dist-test-slave-rmhg"
server_key: "16d786796fdb7a1c0d3144dd8c38f4c1"
server_key_iv: "56b063123a614591933aee636942ea7b"
server_key_version: "encryptionkey@0"
I20260811 12:21:22.998221 10901 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:21:23.002348 10929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:23.004041 10901 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:21:23.004241 10901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "25cfe566171b411aa65d169c7a3b1f9f"
format_stamp: "Formatted at 2026-08-11 12:21:22 on dist-test-slave-rmhg"
server_key: "16d786796fdb7a1c0d3144dd8c38f4c1"
server_key_iv: "56b063123a614591933aee636942ea7b"
server_key_version: "encryptionkey@0"
I20260811 12:21:23.004456 10901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:23.033447 10901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:23.038136 10901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:23.038410 10901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:23.057201 10901 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:38921
I20260811 12:21:23.057250 11012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:38921 every 8 connection(s)
I20260811 12:21:23.058660 10901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:23.062574 11013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:23.063243 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 10901
I20260811 12:21:23.063411 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:23.063850 7324 external_mini_cluster.cc:1428] Setting key 3cfdac5345f15036271b6ef7a612deeb
I20260811 12:21:23.076740 11013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f: Bootstrap starting.
I20260811 12:21:23.078555 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:23.065780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51389 (local address 127.7.39.62:38921)
0811 12:21:23.066622 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:23.066635 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:23.066682 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:21:23.068851 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:23.068884 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:23.069404 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:23.069789 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:23.070423 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:23.071763 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:23.073780 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:23.074582 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:23.074740 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:23.076081 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:23.076089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:23.076105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:23.076688 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:23.076781 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:23.076808 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:23.076929 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:23.077224 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:21:23.077517 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":356,"threads_started":1}
I20260811 12:21:23.080231 11013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:23.081535 11013 log.cc:824] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:23.082535 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:23.084717 11013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f: No bootstrap required, opened a new log
I20260811 12:21:23.093581 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:23.087215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36435 (local address 127.7.39.62:38921)
0811 12:21:23.087446 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:23.087454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:23.087808 (+ 354us) server_negotiation.cc:407] Connection header received
0811 12:21:23.087992 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:23.087996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:23.088071 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:23.088163 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:23.088742 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:23.089561 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:23.090945 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:23.091421 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:23.091550 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:23.092777 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:23.092783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:23.092786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:23.092808 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:23.092853 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:23.092869 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:23.092951 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:23.093093 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 12:21:23.093224 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:21:23.093389 11013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cfe566171b411aa65d169c7a3b1f9f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38921 } }
I20260811 12:21:23.093969 11013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:23.094081 11013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25cfe566171b411aa65d169c7a3b1f9f, State: Initialized, Role: FOLLOWER
I20260811 12:21:23.094838 11013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cfe566171b411aa65d169c7a3b1f9f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38921 } }
I20260811 12:21:23.095155 11013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:23.095295 11013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:23.095459 11013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:23.097980 11013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cfe566171b411aa65d169c7a3b1f9f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38921 } }
I20260811 12:21:23.098572 11013 leader_election.cc:302] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25cfe566171b411aa65d169c7a3b1f9f; no voters:
I20260811 12:21:23.099195 11013 leader_election.cc:288] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:23.099335 11026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:23.100028 11026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [term 1 LEADER]: Becoming Leader. State: Replica: 25cfe566171b411aa65d169c7a3b1f9f, State: Running, Role: LEADER
I20260811 12:21:23.100625 11026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cfe566171b411aa65d169c7a3b1f9f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38921 } }
I20260811 12:21:23.101436 11013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:23.103618 11028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25cfe566171b411aa65d169c7a3b1f9f. Latest consensus state: current_term: 1 leader_uuid: "25cfe566171b411aa65d169c7a3b1f9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cfe566171b411aa65d169c7a3b1f9f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38921 } } }
I20260811 12:21:23.104029 11028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:23.104579 11027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25cfe566171b411aa65d169c7a3b1f9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cfe566171b411aa65d169c7a3b1f9f" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38921 } } }
I20260811 12:21:23.104872 11027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:23.110142 11036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:23.115393 11036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:23.125855 11036 catalog_manager.cc:1347] Generated new cluster ID: d01c662695eb46cda6a605f74f84ad61
I20260811 12:21:23.126016 11036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:23.143451 11036 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:23.145623 11036 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:23.163499 11036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25cfe566171b411aa65d169c7a3b1f9f: Generated new TSK 0
I20260811 12:21:23.164875 11036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:23.177778 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10901
W20260811 12:21:23.179190 11008 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51389
W20260811 12:21:23.180052 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 10901: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:21:23.180238 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 10901
2026-08-11T12:21:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:23 dist-test-slave-rmhg krb5kdc[11057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:23 dist-test-slave-rmhg krb5kdc[11057](info): set up 2 sockets
Aug 11 12:21:23 dist-test-slave-rmhg krb5kdc[11057](info): commencing operation
krb5kdc: starting...
W20260811 12:21:24.460878 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.001s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:24 dist-test-slave-rmhg krb5kdc[11057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:21:24.814451 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35291
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:35291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:25.181318 11073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:25.181686 11073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:25.181836 11073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:25.189536 11073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:25.189684 11073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:25.189769 11073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:25.189853 11073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:25.189915 11073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:25.200802 11073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:35291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:35291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:25.202360 11073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:25.203869 11073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:25.217362 11085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:25.217456 11082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:25.217485 11080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:25.218606 11073 server_base.cc:1034] running on GCE node
I20260811 12:21:25.219550 11073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:25.221275 11073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:25.222550 11073 hybrid_clock.cc:648] HybridClock initialized: now 1786450885222479 us; error 68 us; skew 500 ppm
I20260811 12:21:25.223021 11073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:25 dist-test-slave-rmhg krb5kdc[11057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450885, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:25.226496 11073 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:21:25.228322 11073 webserver.cc:460] Webserver started at http://127.7.39.62:41185/ using document root <none> and password file <none>
I20260811 12:21:25.229097 11073 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:25.229285 11073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:25.229627 11073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:25.232882 11073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "5070390f12ad4860987bfba68a3acbca"
format_stamp: "Formatted at 2026-08-11 12:21:25 on dist-test-slave-rmhg"
server_key: "af227e659fc57749b08b92165cb74743"
server_key_iv: "7ca383528f62086e0cdb296d16544e28"
server_key_version: "encryptionkey@0"
I20260811 12:21:25.233753 11073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "5070390f12ad4860987bfba68a3acbca"
format_stamp: "Formatted at 2026-08-11 12:21:25 on dist-test-slave-rmhg"
server_key: "af227e659fc57749b08b92165cb74743"
server_key_iv: "7ca383528f62086e0cdb296d16544e28"
server_key_version: "encryptionkey@0"
I20260811 12:21:25.239537 11073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:21:25.243753 11096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:25.245723 11073 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:21:25.245980 11073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "5070390f12ad4860987bfba68a3acbca"
format_stamp: "Formatted at 2026-08-11 12:21:25 on dist-test-slave-rmhg"
server_key: "af227e659fc57749b08b92165cb74743"
server_key_iv: "7ca383528f62086e0cdb296d16544e28"
server_key_version: "encryptionkey@0"
I20260811 12:21:25.246245 11073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:25.253636 11073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:25.258558 11073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:25.258895 11073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:25.278492 11073 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:35291
I20260811 12:21:25.278538 11176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:35291 every 8 connection(s)
I20260811 12:21:25.280164 11073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:25.284539 11179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:25.290422 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11073
I20260811 12:21:25.290658 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:25.291219 7324 external_mini_cluster.cc:1428] Setting key 8508544fb5ef5d639aa1b83c769d6d69
I20260811 12:21:25.298062 11179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca: Bootstrap starting.
I20260811 12:21:25.300663 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:25.293574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45625 (local address 127.7.39.62:35291)
0811 12:21:25.294555 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:25.294569 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:25.294638 (+ 69us) server_negotiation.cc:407] Connection header received
0811 12:21:25.296684 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:25.296721 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:25.297183 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:25.297659 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:25.298879 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:25.298890 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:25.298905 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:25.298925 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:25.298993 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:25.299017 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:25.299245 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:25.299410 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 12:21:25.299628 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":446,"threads_started":1}
I20260811 12:21:25.302255 11179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:25.303475 11179 log.cc:824] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:25.305101 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:25.307153 11179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca: No bootstrap required, opened a new log
I20260811 12:21:25.313397 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:25.310792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36411 (local address 127.7.39.62:35291)
0811 12:21:25.311004 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:25.311010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:25.311242 (+ 232us) server_negotiation.cc:407] Connection header received
0811 12:21:25.311423 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:25.311428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:25.311512 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:25.311607 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:25.312661 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:25.312669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:25.312672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:21:25.312695 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:25.312742 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:21:25.312758 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:25.312867 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:25.312998 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 12:21:25.313056 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:21:25.316985 11179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070390f12ad4860987bfba68a3acbca" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35291 } }
I20260811 12:21:25.317500 11179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:21:25.317625 11179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5070390f12ad4860987bfba68a3acbca, State: Initialized, Role: FOLLOWER
I20260811 12:21:25.318310 11179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070390f12ad4860987bfba68a3acbca" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35291 } }
I20260811 12:21:25.318594 11179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:21:25.318744 11179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:21:25.318928 11179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:25.321671 11179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070390f12ad4860987bfba68a3acbca" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35291 } }
I20260811 12:21:25.322254 11179 leader_election.cc:302] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5070390f12ad4860987bfba68a3acbca; no voters:
I20260811 12:21:25.322954 11179 leader_election.cc:288] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:21:25.323127 11196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:25.324329 11196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [term 1 LEADER]: Becoming Leader. State: Replica: 5070390f12ad4860987bfba68a3acbca, State: Running, Role: LEADER
I20260811 12:21:25.325076 11196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070390f12ad4860987bfba68a3acbca" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35291 } }
I20260811 12:21:25.325548 11179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:25.328663 11200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5070390f12ad4860987bfba68a3acbca. Latest consensus state: current_term: 1 leader_uuid: "5070390f12ad4860987bfba68a3acbca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070390f12ad4860987bfba68a3acbca" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35291 } } }
I20260811 12:21:25.329185 11200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:25.329821 11199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5070390f12ad4860987bfba68a3acbca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5070390f12ad4860987bfba68a3acbca" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 35291 } } }
I20260811 12:21:25.330094 11199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:25.334736 11207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:25.340572 11207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:21:25.350714 11207 catalog_manager.cc:1347] Generated new cluster ID: b7166cf5188c419e88b9fbd09f21b0fa
I20260811 12:21:25.350855 11207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:25.374073 11207 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:25.374570 11207 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:25.395159 11207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5070390f12ad4860987bfba68a3acbca: Generated new TSK 0
I20260811 12:21:25.396382 11207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:21:25.410861 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11073
W20260811 12:21:25.412164 11172 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45625
W20260811 12:21:25.412966 7324 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11073: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:21:25.413175 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11073
2026-08-11T12:21:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2232 ms)
[----------] 18 tests from AuthTokenIssuingTest (40739 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:25 dist-test-slave-rmhg krb5kdc[11228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:25 dist-test-slave-rmhg krb5kdc[11228](info): set up 2 sockets
Aug 11 12:21:25 dist-test-slave-rmhg krb5kdc[11228](info): commencing operation
krb5kdc: starting...
W20260811 12:21:26.700620 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.003s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:26 dist-test-slave-rmhg krb5kdc[11228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:21:27.065227 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45533
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:45533,127.7.39.61:43447,127.7.39.60:44385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:27.432154 11243 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:21:27.432533 11243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:27.432646 11243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:27.432761 11243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:27.440239 11243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:27.440351 11243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:27.440421 11243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:27.440493 11243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:27.440577 11243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:27.451570 11243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:45533,127.7.39.61:43447,127.7.39.60:44385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:45533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:27.453002 11243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:27.454444 11243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:27.466282 11250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:27.466349 11251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:27.466311 11254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:27.467242 11243 server_base.cc:1034] running on GCE node
I20260811 12:21:27.468046 11243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:27.469599 11243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:27.470832 11243 hybrid_clock.cc:648] HybridClock initialized: now 1786450887470766 us; error 46 us; skew 500 ppm
I20260811 12:21:27.471192 11243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:27 dist-test-slave-rmhg krb5kdc[11228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:27.474556 11243 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:21:27.476115 11243 webserver.cc:460] Webserver started at http://127.7.39.62:42477/ using document root <none> and password file <none>
I20260811 12:21:27.476872 11243 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:27.476986 11243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:27.477361 11243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:27.480607 11243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "01ab9620f696409caff59e18864990a6"
format_stamp: "Formatted at 2026-08-11 12:21:27 on dist-test-slave-rmhg"
server_key: "65c3a5e1109e79d8f48f3134f925b0dd"
server_key_iv: "7743314fdb368cdf8ac0921e476d90b1"
server_key_version: "encryptionkey@0"
I20260811 12:21:27.481523 11243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "01ab9620f696409caff59e18864990a6"
format_stamp: "Formatted at 2026-08-11 12:21:27 on dist-test-slave-rmhg"
server_key: "65c3a5e1109e79d8f48f3134f925b0dd"
server_key_iv: "7743314fdb368cdf8ac0921e476d90b1"
server_key_version: "encryptionkey@0"
I20260811 12:21:27.487326 11243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:21:27.491405 11263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:27.492995 11243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:21:27.493247 11243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "01ab9620f696409caff59e18864990a6"
format_stamp: "Formatted at 2026-08-11 12:21:27 on dist-test-slave-rmhg"
server_key: "65c3a5e1109e79d8f48f3134f925b0dd"
server_key_iv: "7743314fdb368cdf8ac0921e476d90b1"
server_key_version: "encryptionkey@0"
I20260811 12:21:27.493499 11243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:27.507962 11243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:27.512776 11243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:27.513073 11243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:27.530716 11243 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:45533
I20260811 12:21:27.530736 11351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:45533 every 8 connection(s)
I20260811 12:21:27.532215 11243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:27.536098 11352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:27.542151 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11243
I20260811 12:21:27.542397 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:27.543296 7324 external_mini_cluster.cc:1428] Setting key 4fe98fcb3ab453f2dea51b1ed30f9af7
I20260811 12:21:27.543509 11352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:27 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:27.570629 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.547550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45533 (local address 127.0.0.1:44788)
0811 12:21:27.548448 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.548500 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:27.550693 (+ 2193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:27.551732 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:27.551737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:27.552330 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:27.552689 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:27.552709 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:27.554919 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:27.554924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:27.556177 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:27.556193 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:27.557510 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:27.557516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:27.557684 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:27.558455 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:27.558484 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:27.561234 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:27.565447 (+ 4213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:27.565460 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:27.565473 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:27.565947 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:27.566552 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:27.566557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:27.566559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:27.566763 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:27.567919 (+ 1156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:27.567926 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:27.568145 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 12:21:27.568426 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 12:21:27.569222 (+ 796us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":186,"thread_start_us":346,"threads_started":1}
I20260811 12:21:27.570652 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.548960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.7.39.62:45533)
0811 12:21:27.549675 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:27.549683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:27.549740 (+ 57us) server_negotiation.cc:407] Connection header received
0811 12:21:27.551151 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:27.551159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:27.551481 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:27.551704 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:27.552922 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:27.554702 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:27.556351 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:27.557246 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:27.557443 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:27.561536 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:27.561548 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:27.561568 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:27.561608 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:27.565266 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:27.566127 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:27.566137 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:27.566140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:27.566399 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:27.566880 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:27.566884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:27.566887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:27.567734 (+ 847us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:27.567964 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:27.568663 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 12:21:27.569001 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":342,"threads_started":1}
I20260811 12:21:27.574067 11352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:21:27.575340 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.574895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44416)
0811 12:21:27.575094 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.575215 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 12:21:27.575727 11268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
I20260811 12:21:27.577512 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.577077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44418)
0811 12:21:27.577282 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.577371 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:21:27.577733 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:43447: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.39.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab.
Entry for principal kudu/127.7.39.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab.
I20260811 12:21:27.623705 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.61:43447
--webserver_interface=127.7.39.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab
--principal=kudu/127.7.39.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:45533,127.7.39.61:43447,127.7.39.60:44385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:21:27.627094 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } attempt: 1
I20260811 12:21:27.628295 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.627840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44432)
0811 12:21:27.628035 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.628127 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:21:27.629307 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.628925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44436)
0811 12:21:27.629043 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.629114 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 12:21:27.629499 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:43447: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
I20260811 12:21:27.697726 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } attempt: 2
I20260811 12:21:27.699011 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.698464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44448)
0811 12:21:27.698744 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.698844 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:21:27.700049 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.699684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44462)
0811 12:21:27.699804 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.699889 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 12:21:27.700254 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:43447: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
I20260811 12:21:27.791499 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } attempt: 3
I20260811 12:21:27.792896 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.792376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44472)
0811 12:21:27.792639 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.792743 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 12:21:27.794123 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.793736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44488)
0811 12:21:27.793880 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.793963 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:21:27.794327 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:43447: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
I20260811 12:21:27.937587 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } attempt: 4
I20260811 12:21:27.938777 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.938240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44492)
0811 12:21:27.938554 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.938644 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260811 12:21:27.939981 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:27.939621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44494)
0811 12:21:27.939754 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:27.939844 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:21:27.940217 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:43447: Network error: Client connection negotiation failed: client connection to 127.7.39.61:43447: connect: Connection refused (error 111)
W20260811 12:21:27.986481 11363 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:21:27.986864 11363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:27.986943 11363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:27.987025 11363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:27.994637 11363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:27.994750 11363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:27.994818 11363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:27.994870 11363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:27.994935 11363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:28.005544 11363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:45533,127.7.39.61:43447,127.7.39.60:44385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab
--principal=kudu/127.7.39.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.61:43447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.39.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:28.006973 11363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:28.008363 11363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:28.020649 11373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:28.020677 11378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:28.020727 11371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:28.021086 11363 server_base.cc:1034] running on GCE node
I20260811 12:21:28.022065 11363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:28.023476 11363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:28.024698 11363 hybrid_clock.cc:648] HybridClock initialized: now 1786450888024626 us; error 45 us; skew 500 ppm
I20260811 12:21:28.025025 11363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450888, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:28.035995 11363 init.cc:377] Logged in from keytab as kudu/127.7.39.61@KRBTEST.COM (short username kudu)
I20260811 12:21:28.037642 11363 webserver.cc:460] Webserver started at http://127.7.39.61:34611/ using document root <none> and password file <none>
I20260811 12:21:28.038369 11363 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:28.038478 11363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:28.038805 11363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:28.041997 11363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/data/instance:
uuid: "5e603e5bcc7f4e369c273bc9c4475894"
format_stamp: "Formatted at 2026-08-11 12:21:28 on dist-test-slave-rmhg"
server_key: "4213cd0c2f1ee67f8cd4dca0ede0232a"
server_key_iv: "fb59f857029c8b9e80b02cb75d79a1b5"
server_key_version: "encryptionkey@0"
I20260811 12:21:28.042846 11363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/wal/instance:
uuid: "5e603e5bcc7f4e369c273bc9c4475894"
format_stamp: "Formatted at 2026-08-11 12:21:28 on dist-test-slave-rmhg"
server_key: "4213cd0c2f1ee67f8cd4dca0ede0232a"
server_key_iv: "fb59f857029c8b9e80b02cb75d79a1b5"
server_key_version: "encryptionkey@0"
I20260811 12:21:28.048745 11363 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 12:21:28.052515 11391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:28.054069 11363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:21:28.054265 11363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/wal
uuid: "5e603e5bcc7f4e369c273bc9c4475894"
format_stamp: "Formatted at 2026-08-11 12:21:28 on dist-test-slave-rmhg"
server_key: "4213cd0c2f1ee67f8cd4dca0ede0232a"
server_key_iv: "fb59f857029c8b9e80b02cb75d79a1b5"
server_key_version: "encryptionkey@0"
I20260811 12:21:28.054488 11363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:28.074491 11363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:28.078910 11363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:28.079185 11363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:28.098898 11363 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.61:43447
I20260811 12:21:28.098941 11467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.61:43447 every 8 connection(s)
I20260811 12:21:28.100373 11363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
I20260811 12:21:28.104789 11468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:28.110272 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11363
I20260811 12:21:28.110512 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-1/wal/instance
I20260811 12:21:28.111075 7324 external_mini_cluster.cc:1428] Setting key 6839e7260534cc55a6fef68ac7ca0900
I20260811 12:21:28.113171 11468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450888, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:28.139353 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.117056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.7.39.62:45533)
0811 12:21:28.117613 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.117620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.118779 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 12:21:28.121503 (+ 2724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.121509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.121593 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.121701 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.124124 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.124908 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.127340 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.127802 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.127944 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.132536 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.132543 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.132547 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.132590 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.135466 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.136571 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.136576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.136579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.136736 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.137311 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.137316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.137319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.137662 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.137796 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.138832 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.139060 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":285,"threads_started":1}
I20260811 12:21:28.140743 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.117664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45533 (local address 127.0.0.1:44798)
0811 12:21:28.118542 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.118599 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:28.121043 (+ 2444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:28.122060 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:28.122074 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:28.122897 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:28.123805 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.123835 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.125130 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.125136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.127099 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.127120 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.128091 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.128100 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.128364 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.129185 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:28.129222 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:28.132206 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:28.135663 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.135674 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.135695 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.136380 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.136860 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.136865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.136869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.137131 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.137816 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:28.137824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:28.138342 (+ 518us) client_negotiation.cc:769] Sending connection context
0811 12:21:28.138661 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 12:21:28.139307 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":329,"threads_started":1}
I20260811 12:21:28.142867 11468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.39.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.60@KRBTEST.COM" created.
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450888, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:21:28.158619 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.143727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44504)
0811 12:21:28.143918 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.143940 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:28.144069 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:28.144926 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:28.144930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:28.145422 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:28.145564 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.145574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.147005 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.147010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.147906 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.147920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.148778 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.148783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.148968 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.149600 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:28.149634 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:28.151900 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:28.155593 (+ 3693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.155599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.155603 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.156142 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.156585 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.156590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.156593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.156752 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.157536 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:28.157541 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:28.157741 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 12:21:28.157867 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 12:21:28.158171 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 12:21:28.158684 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.143839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44504 (local address 127.7.39.61:43447)
0811 12:21:28.144476 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.144483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.144508 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:21:28.144557 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.144563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.144764 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.144924 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.145741 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.146835 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.148043 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.148628 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.148746 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.152233 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.152250 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.152270 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.152305 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.155401 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.156312 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.156321 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.156323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.156465 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.156868 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.156872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.156874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.157386 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.157620 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.158062 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.158319 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":331,"threads_started":1}
I20260811 12:21:28.161684 11468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:21:28.162959 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.162462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42058)
0811 12:21:28.162659 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.162788 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 12:21:28.163247 11397 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.167129 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.166635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42068)
0811 12:21:28.166866 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.166972 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 12:21:28.167423 11468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab.
Entry for principal kudu/127.7.39.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab.
I20260811 12:21:28.194255 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.60:44385
--webserver_interface=127.7.39.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab
--principal=kudu/127.7.39.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:45533,127.7.39.61:43447,127.7.39.60:44385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:21:28.216835 11468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 1
I20260811 12:21:28.218182 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.217730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42084)
0811 12:21:28.217926 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.218021 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 12:21:28.219161 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.218797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42098)
0811 12:21:28.218949 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.219046 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:21:28.219372 11468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.239459 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } attempt: 5
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:21:28.255056 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.240276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44506)
0811 12:21:28.240493 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.240514 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:28.240674 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:28.241030 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:28.241034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:28.241495 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:28.241729 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.241750 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.243204 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.243208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.244100 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.244113 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.245194 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.245202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.245388 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.246131 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:28.246151 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:28.248581 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:28.252590 (+ 4009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.252595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.252598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.253008 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.253469 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.253472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.253475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.253618 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.254289 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:28.254292 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:28.254422 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 12:21:28.254539 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:21:28.254805 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 12:21:28.255376 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.240323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.7.39.61:43447)
0811 12:21:28.240521 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.240526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.240595 (+ 69us) server_negotiation.cc:407] Connection header received
0811 12:21:28.240800 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.240805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.240887 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.240975 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.241932 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.243008 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.244271 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.244952 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.245097 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.248866 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.248875 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.248880 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.248928 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.252402 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.253181 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.253188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.253192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.253361 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.253738 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.253743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.253746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.254178 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.254358 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.254673 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.254933 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 12:21:28.256381 11352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:21:28.257524 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.257091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42106)
0811 12:21:28.257291 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.257374 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 12:21:28.258567 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.258185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42114)
0811 12:21:28.258351 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.258459 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 12:21:28.258759 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.287631 11468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 2
I20260811 12:21:28.288964 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.288420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42128)
0811 12:21:28.288671 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.288787 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 12:21:28.290060 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.289694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42132)
0811 12:21:28.289850 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.289929 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:21:28.290294 11468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.310015 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 1
I20260811 12:21:28.311290 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.310778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42134)
0811 12:21:28.310987 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.311085 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 12:21:28.312359 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.312004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42142)
0811 12:21:28.312165 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.312244 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:21:28.312584 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.380798 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 2
I20260811 12:21:28.381497 11468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 3
I20260811 12:21:28.382547 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.381650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42158)
0811 12:21:28.382173 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.382320 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301}
I20260811 12:21:28.382702 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.382110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42172)
0811 12:21:28.382412 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.382504 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260811 12:21:28.383967 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.383440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42188)
0811 12:21:28.383740 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.383831 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
W20260811 12:21:28.384219 11468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.385365 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.383415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42174)
0811 12:21:28.384876 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.385046 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1333,"threads_started":1}
W20260811 12:21:28.388651 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.494921 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 3
I20260811 12:21:28.496126 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.495672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42198)
0811 12:21:28.495877 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.495972 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 12:21:28.497419 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.496902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42214)
0811 12:21:28.497120 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.497254 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260811 12:21:28.497627 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.527470 11468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 4
I20260811 12:21:28.528792 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.528271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42228)
0811 12:21:28.528513 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.528613 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 12:21:28.529868 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.529510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42240)
0811 12:21:28.529673 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.529751 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 12:21:28.530130 11468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
W20260811 12:21:28.578362 11483 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:21:28.578737 11483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:28.578845 11483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:28.578958 11483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:28.586650 11483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:28.586771 11483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:28.586830 11483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:28.586902 11483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:28.586987 11483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:28.597815 11483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:45533,127.7.39.61:43447,127.7.39.60:44385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab
--principal=kudu/127.7.39.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.60:44385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.39.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:28.599331 11483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:28.600790 11483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:28.613901 11489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:28.613823 11493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:28.613873 11490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:28.615175 11483 server_base.cc:1034] running on GCE node
I20260811 12:21:28.616026 11483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:28.617697 11483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:28.618930 11483 hybrid_clock.cc:648] HybridClock initialized: now 1786450888618856 us; error 70 us; skew 500 ppm
I20260811 12:21:28.619254 11483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450888, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:28.622503 11483 init.cc:377] Logged in from keytab as kudu/127.7.39.60@KRBTEST.COM (short username kudu)
I20260811 12:21:28.624075 11483 webserver.cc:460] Webserver started at http://127.7.39.60:35907/ using document root <none> and password file <none>
I20260811 12:21:28.624809 11483 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:28.624902 11483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:28.625239 11483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:28.628301 11483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/data/instance:
uuid: "7871221bf8af4b54b03490c921ac641d"
format_stamp: "Formatted at 2026-08-11 12:21:28 on dist-test-slave-rmhg"
server_key: "f0941cf4c2f10213bc3a92980e52551e"
server_key_iv: "226128a433a3a97761d4ef15eee41067"
server_key_version: "encryptionkey@0"
I20260811 12:21:28.629087 11483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/wal/instance:
uuid: "7871221bf8af4b54b03490c921ac641d"
format_stamp: "Formatted at 2026-08-11 12:21:28 on dist-test-slave-rmhg"
server_key: "f0941cf4c2f10213bc3a92980e52551e"
server_key_iv: "226128a433a3a97761d4ef15eee41067"
server_key_version: "encryptionkey@0"
I20260811 12:21:28.634490 11483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:21:28.638012 11503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:28.639392 11483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:21:28.639544 11483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/wal
uuid: "7871221bf8af4b54b03490c921ac641d"
format_stamp: "Formatted at 2026-08-11 12:21:28 on dist-test-slave-rmhg"
server_key: "f0941cf4c2f10213bc3a92980e52551e"
server_key_iv: "226128a433a3a97761d4ef15eee41067"
server_key_version: "encryptionkey@0"
I20260811 12:21:28.639806 11483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:28.656462 11483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:28.661022 11483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:28.661341 11483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:28.674830 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 4
I20260811 12:21:28.675882 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.675455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42242)
0811 12:21:28.675649 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.675756 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:21:28.677018 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.676616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42250)
0811 12:21:28.676766 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.676860 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:21:28.677289 11352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:44385: Network error: Client connection negotiation failed: client connection to 127.7.39.60:44385: connect: Connection refused (error 111)
I20260811 12:21:28.680508 11483 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.60:44385
I20260811 12:21:28.680533 11579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.60:44385 every 8 connection(s)
I20260811 12:21:28.682049 11483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
I20260811 12:21:28.686023 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:28.690744 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11483
I20260811 12:21:28.690953 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450508543501-7324-0/minicluster-data/master-2/wal/instance
I20260811 12:21:28.691516 7324 external_mini_cluster.cc:1428] Setting key dabe36dee8db28399610b8b224787f34
I20260811 12:21:28.694221 11580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450888, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:28.714151 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.698409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.7.39.62:45533)
0811 12:21:28.698971 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.698977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.699475 (+ 498us) server_negotiation.cc:407] Connection header received
0811 12:21:28.699693 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.699698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.699765 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.699846 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.701539 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.702243 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.704089 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.704713 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.704855 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.708385 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.708390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.708393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.708430 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.710933 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.711685 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.711691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.711695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.711847 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.712242 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.712247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.712251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.712546 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.712769 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.713467 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.713711 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":230,"threads_started":1}
I20260811 12:21:28.715378 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.693719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.7.39.60:44385)
0811 12:21:28.694581 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.694595 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.694651 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:21:28.696340 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.696368 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.696841 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.697250 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.699214 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.701289 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.702626 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.703672 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.703867 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.708054 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.708066 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.708085 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.708126 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.711025 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.711718 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.711724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.711726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.711875 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.712251 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.712254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.712256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.712922 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.713344 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.713615 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.713997 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":212,"server-negotiator.queue_time_us":633,"thread_start_us":399,"threads_started":1}
I20260811 12:21:28.715411 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.698463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:45533 (local address 127.0.0.1:44808)
0811 12:21:28.699292 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.699347 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:28.699545 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:28.699946 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:28.699954 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:28.700854 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:28.701267 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.701287 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.702397 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.702402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.703926 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.703939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.704966 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.704973 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.705121 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.705724 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:28.705747 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:28.708188 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:28.711080 (+ 2892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.711092 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.711107 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.711542 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.711960 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.711965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.711968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.712126 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.712691 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:28.712699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:28.712949 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 12:21:28.713344 (+ 395us) client_negotiation.cc:240] Negotiation successful
0811 12:21:28.713873 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":2,"thread_start_us":427,"threads_started":1}
I20260811 12:21:28.717131 11580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450888, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:28.732653 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.718086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:43447 (local address 127.0.0.1:44518)
0811 12:21:28.718249 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.718267 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:28.718381 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:28.718910 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:28.718915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:28.719384 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:28.719547 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.719556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.720865 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.720871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.722067 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.722079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.723031 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.723038 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.723229 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.723861 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:28.723883 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:28.726222 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:28.729997 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.730022 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.730026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.730509 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.730944 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.730949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.730951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.731104 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.731657 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:28.731663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:28.731828 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 12:21:28.731953 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 12:21:28.732290 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:21:28.732702 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.718058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44518 (local address 127.7.39.61:43447)
0811 12:21:28.718238 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.718244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.718331 (+ 87us) server_negotiation.cc:407] Connection header received
0811 12:21:28.718539 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.718545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.718630 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.718724 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.719710 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.720712 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.722207 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.722855 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.722991 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.726496 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.726505 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.726510 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.726564 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.729812 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.730656 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.730662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.730666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.730818 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.731223 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.731229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.731232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.731521 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.731689 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.732069 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.732298 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:21:28.733948 11580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:21:28.734195 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.718694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.7.39.62:45533)
0811 12:21:28.718864 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.718869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.718885 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:21:28.718993 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.718997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.719083 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.719186 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.720546 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.721522 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.723064 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.723657 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.723772 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.728398 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.728404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.728407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.728447 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.731324 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.732053 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.732058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.732060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.732208 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.732618 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.732622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.732624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.732979 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.733303 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.733653 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.733906 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450888, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:21:28.748533 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.734729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.7.39.60:44385)
0811 12:21:28.734891 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.734897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.734947 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:21:28.735124 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.735129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.735223 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.735294 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.736135 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.737212 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.738725 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.739249 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.739380 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.742985 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.742993 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.742997 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.743037 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.745904 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.746698 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.746703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.746707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.746848 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.747248 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.747252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.747254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.747542 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.747719 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.748099 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.748283 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:21:28.748881 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.734687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42272)
0811 12:21:28.734877 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.734898 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:28.735016 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:28.735338 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:28.735342 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:28.735795 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:28.735982 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.735996 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.737337 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.737341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.738544 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.738560 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.739403 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.739408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.739578 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.740156 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:28.740177 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:28.742739 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:28.746041 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.746046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.746050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.746555 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.746963 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.746968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.746972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.747142 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.747674 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:28.747680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:28.747851 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 12:21:28.748028 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 12:21:28.748495 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":94}
I20260811 12:21:28.751945 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.735888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.7.39.61:43447)
0811 12:21:28.736057 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.736063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.736080 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:21:28.736162 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.736167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.736224 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.736294 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.737769 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.738823 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.740338 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.740968 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.741105 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.745896 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.745902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.745905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.745942 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.749323 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.750165 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.750169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.750171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.750298 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.750737 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.750740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.750742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.751011 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.751215 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.751549 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.751721 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:21:28.754585 11580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: Bootstrap starting.
I20260811 12:21:28.757040 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:28.758139 11580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:28.759279 11580 log.cc:824] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:28.762555 11580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: No bootstrap required, opened a new log
I20260811 12:21:28.759802 11317 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "01ab9620f696409caff59e18864990a6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44824
I20260811 12:21:28.770334 11580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } }
I20260811 12:21:28.770735 11580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7871221bf8af4b54b03490c921ac641d, State: Initialized, Role: FOLLOWER
I20260811 12:21:28.771225 11580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } }
I20260811 12:21:28.772192 11591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:28.772521 11591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:28.773600 11580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:21:28.783674 11613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:21:28.783841 11613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:21:28.781291 11483 master_runner.cc:186] Error getting master registration for 127.7.39.62:45533: OK, instance_id { permanent_uuid: "01ab9620f696409caff59e18864990a6" instance_seqno: 1786450887525558 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:21:28.792706 11483 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:21:28.829401 11468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 5
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450888, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
I20260811 12:21:28.843811 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.830291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.7.39.60:44385)
0811 12:21:28.830495 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.830500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.830526 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:21:28.830746 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.830750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.830820 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.830915 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.831765 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.832522 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.833830 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.834301 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.834426 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.838161 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.838166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.838169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.838206 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.840946 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.841712 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.841717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.841719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.841861 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.842268 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.842271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.842273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.842631 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.842789 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.843184 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.843467 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:21:28.843807 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.830205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42284)
0811 12:21:28.830428 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.830452 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:28.830596 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:28.830967 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:28.830972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:28.831412 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:28.831581 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.831590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.832678 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.832685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.833681 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.833695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.834461 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.834467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.834671 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.835333 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:28.835358 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:28.837959 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:28.841121 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.841127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.841131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.841580 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.841980 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.841985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.841989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.842149 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.842754 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:28.842759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:28.842918 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 12:21:28.843044 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 12:21:28.843370 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 12:21:28.850654 11468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: Bootstrap starting.
I20260811 12:21:28.854903 11468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:28.856276 11468 log.cc:824] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:28.859802 11468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: No bootstrap required, opened a new log
I20260811 12:21:28.871114 11468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } }
I20260811 12:21:28.871578 11468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e603e5bcc7f4e369c273bc9c4475894, State: Initialized, Role: FOLLOWER
I20260811 12:21:28.872169 11468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } }
I20260811 12:21:28.873338 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:28.873764 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:28.874670 11468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:21:28.888718 11633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:21:28.889015 11633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:21:28.882247 11363 master_runner.cc:186] Error getting master registration for 127.7.39.62:45533: OK, instance_id { permanent_uuid: "01ab9620f696409caff59e18864990a6" instance_seqno: 1786450887525558 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:21:28.894786 11363 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 12:21:28.913733 11511 tablet.cc:2369] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:21:28.954560 11352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } attempt: 5
Aug 11 12:21:28 dist-test-slave-rmhg krb5kdc[11228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
I20260811 12:21:28.973197 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.955396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:44385 (local address 127.0.0.1:42294)
0811 12:21:28.955728 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:28.955750 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:28.955894 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:28.956352 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:28.956357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:28.956919 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:28.957184 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.957202 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.958756 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.958762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.960158 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:28.960176 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.961507 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.961513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:28.961728 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.962433 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:28.962459 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:28.965568 (+ 3109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:28.970071 (+ 4503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.970077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.970082 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.970653 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.971142 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:28.971147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:28.971151 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:28.971354 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:28.972101 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:28.972107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:28.972282 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 12:21:28.972431 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 12:21:28.972789 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260811 12:21:28.974148 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:28.955494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42294 (local address 127.7.39.60:44385)
0811 12:21:28.955731 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:28.955738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:28.955828 (+ 90us) server_negotiation.cc:407] Connection header received
0811 12:21:28.956068 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:28.956075 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:28.956174 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:28.956286 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:28.957389 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.958564 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.960350 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:28.961105 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:28.961253 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:28.965844 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:28.965853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:28.965858 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:28.965906 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:28.969830 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.970811 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.970817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.970824 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.970993 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:28.971489 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:28.971494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:28.971497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:28.971884 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:28.972034 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:28.973525 (+ 1491us) server_negotiation.cc:299] Negotiation successful
0811 12:21:28.973787 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 12:21:28.980803 11352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6: Bootstrap starting.
I20260811 12:21:28.985329 11352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:28.986809 11352 log.cc:824] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:28.991510 11352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6: No bootstrap required, opened a new log
I20260811 12:21:29.003811 11352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } }
I20260811 12:21:29.004624 11352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ab9620f696409caff59e18864990a6, State: Initialized, Role: FOLLOWER
I20260811 12:21:29.005704 11352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } }
I20260811 12:21:29.007213 11636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:29.007712 11636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:29.009986 11317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 12:21:29.010224 11317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:29.013301 11352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:29.013870 11317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } }
W20260811 12:21:29.016428 11270 tablet.cc:2369] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:21:29.016512 11433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01ab9620f696409caff59e18864990a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5e603e5bcc7f4e369c273bc9c4475894"
I20260811 12:21:29.017079 11433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:29.020725 11433 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ab9620f696409caff59e18864990a6 in term 1.
I20260811 12:21:29.022020 11268 leader_election.cc:302] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ab9620f696409caff59e18864990a6, 5e603e5bcc7f4e369c273bc9c4475894; no voters:
I20260811 12:21:29.022686 11636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:29.025924 11544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01ab9620f696409caff59e18864990a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7871221bf8af4b54b03490c921ac641d"
I20260811 12:21:29.026445 11544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:29.030408 11544 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ab9620f696409caff59e18864990a6 in term 1.
I20260811 12:21:29.031440 11317 leader_election.cc:288] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [CANDIDATE]: Term 1 election: Requested vote from peers 5e603e5bcc7f4e369c273bc9c4475894 (127.7.39.61:43447), 7871221bf8af4b54b03490c921ac641d (127.7.39.60:44385)
I20260811 12:21:29.039455 11636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [term 1 LEADER]: Becoming Leader. State: Replica: 01ab9620f696409caff59e18864990a6, State: Running, Role: LEADER
I20260811 12:21:29.040285 11636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } }
I20260811 12:21:29.049126 11641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01ab9620f696409caff59e18864990a6. Latest consensus state: current_term: 1 leader_uuid: "01ab9620f696409caff59e18864990a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:29.049750 11641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:29.059300 11646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:29.078359 11646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 12:21:29.082149 11404 tablet.cc:2369] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:21:29.113281 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:29.073582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44826 (local address 127.7.39.62:45533)
0811 12:21:29.073777 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.073784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.085851 (+ 12067us) server_negotiation.cc:407] Connection header received
0811 12:21:29.086227 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.086234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.086332 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.086409 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.098468 (+ 12059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:29.099692 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:29.101818 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:29.102443 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:29.102545 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:29.105760 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:29.105767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:29.105770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:29.105828 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:29.109673 (+ 3845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:29.110691 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:29.110696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:29.110700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:29.110853 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:29.111333 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:29.111339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:29.111343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:29.111754 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:29.112031 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:29.112623 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 12:21:29.112853 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:21:29.116792 11544 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [term 1 FOLLOWER]: Refusing update from remote peer 01ab9620f696409caff59e18864990a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:21:29.117215 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:29.078406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.7.39.60:44385)
0811 12:21:29.078669 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.078676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.090399 (+ 11723us) server_negotiation.cc:407] Connection header received
0811 12:21:29.090607 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.090613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.090708 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.090815 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.096397 (+ 5582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:29.097577 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:29.099731 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:29.100392 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:29.100496 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:29.104495 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:29.104502 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:29.104507 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:29.104560 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:29.108260 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:29.114581 (+ 6321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:29.114587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:29.114590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:29.114799 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:29.115315 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:29.115320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:29.115323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:29.115698 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:29.115974 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:29.116560 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 12:21:29.116833 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:21:29.117930 11641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:21:29.130733 11433 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [term 1 FOLLOWER]: Refusing update from remote peer 01ab9620f696409caff59e18864990a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:21:29.131206 11680 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:21:29.131815 11641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:21:29.133963 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:29.086678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44532 (local address 127.7.39.61:43447)
0811 12:21:29.086869 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.086876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.093841 (+ 6965us) server_negotiation.cc:407] Connection header received
0811 12:21:29.097107 (+ 3266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.097113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.097228 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.097311 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.110209 (+ 12898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:29.111386 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:29.117598 (+ 6212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:29.118338 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:29.118449 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:29.123780 (+ 5331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:29.123789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:29.123794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:29.123850 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:29.128121 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:29.129347 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:29.130072 (+ 725us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:29.130077 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:29.130266 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:29.131466 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:29.131472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:29.131476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:29.132059 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:29.132321 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:29.133135 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 12:21:29.133459 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:21:29.147681 11591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01ab9620f696409caff59e18864990a6. Latest consensus state: current_term: 1 leader_uuid: "01ab9620f696409caff59e18864990a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:29.148032 11591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:29.154361 11682 mvcc.cc:205] Tried to move back new op lower bound from 7317302841771143168 to 7317302841516343296. Current Snapshot: MvccSnapshot[applied={T|T < 7317302841771143168}]
I20260811 12:21:29.155673 11671 mvcc.cc:205] Tried to move back new op lower bound from 7317302841771143168 to 7317302841516343296. Current Snapshot: MvccSnapshot[applied={T|T < 7317302841771143168}]
I20260811 12:21:29.163507 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01ab9620f696409caff59e18864990a6. Latest consensus state: current_term: 1 leader_uuid: "01ab9620f696409caff59e18864990a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:29.163887 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:29.169243 11683 mvcc.cc:205] Tried to move back new op lower bound from 7317302841771143168 to 7317302841516343296. Current Snapshot: MvccSnapshot[applied={T|T < 7317302841771143168}]
I20260811 12:21:29.175377 11591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01ab9620f696409caff59e18864990a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:29.175731 11591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:29.177181 11641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01ab9620f696409caff59e18864990a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:29.177460 11641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:29.177772 11641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01ab9620f696409caff59e18864990a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:29.177958 11641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:29.183233 11646 catalog_manager.cc:1347] Generated new cluster ID: 833adb6ef0484c09a82da45f2726ff85
I20260811 12:21:29.183369 11646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:29.187357 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01ab9620f696409caff59e18864990a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ab9620f696409caff59e18864990a6" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 45533 } } peers { permanent_uuid: "5e603e5bcc7f4e369c273bc9c4475894" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 43447 } } peers { permanent_uuid: "7871221bf8af4b54b03490c921ac641d" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 44385 } } }
I20260811 12:21:29.187577 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:29.251226 11646 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:29.253290 11646 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:29.279114 11646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01ab9620f696409caff59e18864990a6: Generated new TSK 0
I20260811 12:21:29.280239 11646 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:21:29.388049 11585 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.377465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.7.39.62:45533)
0811 12:21:29.377697 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.377704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.385899 (+ 8195us) server_negotiation.cc:407] Connection header received
0811 12:21:29.386243 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.386250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.386347 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.386424 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.387781 (+ 1357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44830: BlockingRecv error: recv got EOF from 127.0.0.1:44830 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 12:21:29.392064 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.377370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.7.39.61:43447)
0811 12:21:29.377554 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.377560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.389819 (+ 12259us) server_negotiation.cc:407] Connection header received
0811 12:21:29.390204 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.390211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.390325 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.390418 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.391771 (+ 1353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44544: BlockingRecv error: recv got EOF from 127.0.0.1:44544 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 12:21:29.392081 11584 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.377061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.7.39.60:44385)
0811 12:21:29.377295 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.377301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.390037 (+ 12736us) server_negotiation.cc:407] Connection header received
0811 12:21:29.390558 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.390565 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.390691 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.390770 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.391786 (+ 1016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42306: BlockingRecv error: recv got EOF from 127.0.0.1:42306 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260811 12:21:29.411011 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.408430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44560 (local address 127.7.39.61:43447)
0811 12:21:29.408647 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.408654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.409233 (+ 579us) server_negotiation.cc:407] Connection header received
0811 12:21:29.409514 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.409520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.409686 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.409834 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.410712 (+ 878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44560: BlockingRecv error: recv got EOF from 127.0.0.1:44560 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 12:21:29.427217 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.424173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.7.39.61:43447)
0811 12:21:29.424356 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.424362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.425567 (+ 1205us) server_negotiation.cc:407] Connection header received
0811 12:21:29.425804 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.425811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.425908 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.426016 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.426972 (+ 956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44568: BlockingRecv error: recv got EOF from 127.0.0.1:44568 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:21:29.439402 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.432502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.7.39.61:43447)
0811 12:21:29.432690 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.432696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.437829 (+ 5133us) server_negotiation.cc:407] Connection header received
0811 12:21:29.438023 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.438029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.438157 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.438271 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.439138 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44576: BlockingRecv error: recv got EOF from 127.0.0.1:44576 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 12:21:29.449748 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.447056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44582 (local address 127.7.39.61:43447)
0811 12:21:29.447355 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.447361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.448068 (+ 707us) server_negotiation.cc:407] Connection header received
0811 12:21:29.448282 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.448288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.448388 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.448511 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.449487 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44582: BlockingRecv error: recv got EOF from 127.0.0.1:44582 (error 108)
Metrics: {"server-negotiator.queue_time_us":168}
W20260811 12:21:29.463434 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.461478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.7.39.61:43447)
0811 12:21:29.461725 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.461731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.462068 (+ 337us) server_negotiation.cc:407] Connection header received
0811 12:21:29.462251 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.462256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.462357 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.462486 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.463197 (+ 711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44592: BlockingRecv error: recv got EOF from 127.0.0.1:44592 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 12:21:29.485030 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.482986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.7.39.61:43447)
0811 12:21:29.483210 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.483216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.483507 (+ 291us) server_negotiation.cc:407] Connection header received
0811 12:21:29.483716 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.483720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.483798 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.483889 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.484782 (+ 893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44606: BlockingRecv error: recv got EOF from 127.0.0.1:44606 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 12:21:29.523970 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.521798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.7.39.61:43447)
0811 12:21:29.522017 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.522023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.522346 (+ 323us) server_negotiation.cc:407] Connection header received
0811 12:21:29.522573 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.522577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.522667 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.522764 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.523690 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44610: BlockingRecv error: recv got EOF from 127.0.0.1:44610 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 12:21:29.594354 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.592244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.7.39.61:43447)
0811 12:21:29.592537 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.592544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.592923 (+ 379us) server_negotiation.cc:407] Connection header received
0811 12:21:29.593129 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.593136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.593265 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.593394 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.594137 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44614: BlockingRecv error: recv got EOF from 127.0.0.1:44614 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 12:21:29.729833 11477 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:29.727777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.7.39.61:43447)
0811 12:21:29.728050 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.728058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.728337 (+ 279us) server_negotiation.cc:407] Connection header received
0811 12:21:29.728572 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.728577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.728660 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.728756 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:29.729635 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58410: BlockingRecv error: recv got EOF from 127.0.0.1:58410 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 12:21:29.785946 11613 catalog_manager.cc:1259] Loaded cluster ID: 833adb6ef0484c09a82da45f2726ff85
I20260811 12:21:29.786067 11613 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: loading cluster ID for follower catalog manager: success
I20260811 12:21:29.788687 11613 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: acquiring CA information for follower catalog manager: success
I20260811 12:21:29.790730 11613 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7871221bf8af4b54b03490c921ac641d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:21:29.891230 11633 catalog_manager.cc:1259] Loaded cluster ID: 833adb6ef0484c09a82da45f2726ff85
I20260811 12:21:29.891342 11633 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: loading cluster ID for follower catalog manager: success
I20260811 12:21:29.893941 11633 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: acquiring CA information for follower catalog manager: success
I20260811 12:21:29.896047 11633 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e603e5bcc7f4e369c273bc9c4475894: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:21:29.996004 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:29.990022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58414 (local address 127.7.39.61:43447)
0811 12:21:29.990295 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:29.990300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:29.990489 (+ 189us) server_negotiation.cc:407] Connection header received
0811 12:21:29.990640 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:29.990644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:29.990808 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:29.990951 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:29.991398 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:29.992164 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:29.993690 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:29.994125 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:29.994232 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:29.994722 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:29.995366 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:29.995484 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:29.995595 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 12:21:29.995675 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 12:21:30.009040 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:30.002516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40568 (local address 127.7.39.60:44385)
0811 12:21:30.003013 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:30.003020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:30.003042 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:21:30.003330 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:30.003334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:30.003493 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:30.003678 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:30.004091 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:30.004914 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:30.006446 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:30.006881 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:30.007019 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:30.007648 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:30.008306 (+ 658us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:30.008465 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:30.008590 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 12:21:30.008702 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":259,"threads_started":1}
I20260811 12:21:30.024471 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:30.016614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.7.39.61:43447)
0811 12:21:30.016861 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:30.016869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:30.018398 (+ 1529us) server_negotiation.cc:407] Connection header received
0811 12:21:30.018723 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:30.018729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:30.018823 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:30.018887 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:30.019860 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:30.020846 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:30.022544 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:30.023040 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:30.023162 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:30.023701 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:30.023853 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:30.023905 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:30.024105 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 12:21:30.024192 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:21:30.025108 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:30.016852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.7.39.60:44385)
0811 12:21:30.017057 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:30.017064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:30.018634 (+ 1570us) server_negotiation.cc:407] Connection header received
0811 12:21:30.018874 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:30.018879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:30.018959 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:30.019020 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:30.019893 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:30.020611 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:30.022958 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:30.023570 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:30.023695 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:30.024340 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:30.024504 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:30.024556 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:30.024778 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 12:21:30.024865 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:21:30.026178 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:30.016196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.7.39.62:45533)
0811 12:21:30.017570 (+ 1374us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:30.017577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:30.018276 (+ 699us) server_negotiation.cc:407] Connection header received
0811 12:21:30.018435 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:30.018440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:30.018713 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:30.019346 (+ 633us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:30.019532 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:30.020571 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:30.023013 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:30.023622 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:30.023749 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:30.024845 (+ 1096us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:30.025394 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:30.025511 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:30.025639 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 12:21:30.025733 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240,"thread_start_us":1098,"threads_started":1}
I20260811 12:21:30.031196 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11243
I20260811 12:21:30.032799 11347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:30.143321 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11243
I20260811 12:21:30.173049 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11363
I20260811 12:21:30.174355 11462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:30.286290 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11363
I20260811 12:21:30.312196 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11483
I20260811 12:21:30.313432 11575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:30.426430 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11483
2026-08-11T12:21:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5014 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:21:30 dist-test-slave-rmhg krb5kdc[11813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:21:30 dist-test-slave-rmhg krb5kdc[11813](info): set up 2 sockets
Aug 11 12:21:30 dist-test-slave-rmhg krb5kdc[11813](info): commencing operation
krb5kdc: starting...
W20260811 12:21:31.706986 7324 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.004s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/test-user.keytab.
Aug 11 12:21:31 dist-test-slave-rmhg krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:21:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:21:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.39.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
Entry for principal kudu/127.7.39.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab.
I20260811 12:21:32.069010 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:38923
--webserver_interface=127.7.39.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:38923,127.7.39.61:33509,127.7.39.60:35675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:21:32.436630 11833 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:21:32.436998 11833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:32.437100 11833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:32.437239 11833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:32.445240 11833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:32.445369 11833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:32.445446 11833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:32.445520 11833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:32.445608 11833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:32.456913 11833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:38923,127.7.39.61:33509,127.7.39.60:35675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.62.keytab
--principal=kudu/127.7.39.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.62:38923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.39.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:32.458491 11833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:32.459975 11833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:32.473124 11848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:32.473074 11844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:32.473094 11843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:32.473858 11833 server_base.cc:1034] running on GCE node
I20260811 12:21:32.475020 11833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:32.476676 11833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:32.477936 11833 hybrid_clock.cc:648] HybridClock initialized: now 1786450892477855 us; error 47 us; skew 500 ppm
I20260811 12:21:32.478323 11833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:32 dist-test-slave-rmhg krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:32.481830 11833 init.cc:377] Logged in from keytab as kudu/127.7.39.62@KRBTEST.COM (short username kudu)
I20260811 12:21:32.483564 11833 webserver.cc:460] Webserver started at http://127.7.39.62:34275/ using document root <none> and password file <none>
I20260811 12:21:32.484373 11833 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:32.484512 11833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:32.484854 11833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:32.488314 11833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/data/instance:
uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7"
format_stamp: "Formatted at 2026-08-11 12:21:32 on dist-test-slave-rmhg"
server_key: "519e6d9b2c5cc8e2f5debd7e868ad933"
server_key_iv: "c3c22771b993b5af23d40ca5817c7d61"
server_key_version: "encryptionkey@0"
I20260811 12:21:32.489287 11833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/wal/instance:
uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7"
format_stamp: "Formatted at 2026-08-11 12:21:32 on dist-test-slave-rmhg"
server_key: "519e6d9b2c5cc8e2f5debd7e868ad933"
server_key_iv: "c3c22771b993b5af23d40ca5817c7d61"
server_key_version: "encryptionkey@0"
I20260811 12:21:32.495054 11833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:21:32.498968 11862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:32.500464 11833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:21:32.500679 11833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/wal
uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7"
format_stamp: "Formatted at 2026-08-11 12:21:32 on dist-test-slave-rmhg"
server_key: "519e6d9b2c5cc8e2f5debd7e868ad933"
server_key_iv: "c3c22771b993b5af23d40ca5817c7d61"
server_key_version: "encryptionkey@0"
I20260811 12:21:32.500909 11833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:32.527311 11833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:32.532261 11833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:32.532569 11833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:32.551761 11833 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.62:38923
I20260811 12:21:32.551774 11943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.62:38923 every 8 connection(s)
I20260811 12:21:32.553262 11833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/data/info.pb
I20260811 12:21:32.555653 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11833
I20260811 12:21:32.555862 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-0/wal/instance
I20260811 12:21:32.556466 7324 external_mini_cluster.cc:1428] Setting key 7bb447b10676e2c8dff49754aca0f319
I20260811 12:21:32.558074 11945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:32.567725 11945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:32 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:32.596947 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.572316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:38923 (local address 127.0.0.1:46020)
0811 12:21:32.574028 (+ 1712us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.574101 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:32.576381 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:32.577628 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:32.577633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:32.578343 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:32.578743 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:32.578765 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:32.581286 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:32.581290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:32.582586 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:32.582603 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:32.583984 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:32.583991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:32.584135 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:32.584897 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:32.584924 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:32.587518 (+ 2594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:32.592126 (+ 4608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:32.592140 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:32.592152 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:32.592777 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:32.593387 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:32.593390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:32.593393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:32.593573 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:32.594731 (+ 1158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:32.594739 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:32.594968 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 12:21:32.595220 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 12:21:32.595706 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"spinlock_wait_cycles":42752,"thread_start_us":1115,"threads_started":1}
I20260811 12:21:32.596997 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.572316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.7.39.62:38923)
0811 12:21:32.575052 (+ 2736us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:32.575061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:32.575126 (+ 65us) server_negotiation.cc:407] Connection header received
0811 12:21:32.576886 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:32.576896 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:32.577332 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:32.577570 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:32.579008 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:32.580931 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:32.582783 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:32.583688 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:32.583896 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:32.587850 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:32.587863 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:32.587884 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:32.587936 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:32.591903 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:32.592959 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:32.592969 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:32.592974 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:32.593248 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:32.593687 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:32.593692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:32.593696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:32.594519 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:32.594764 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:32.595431 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 12:21:32.595915 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":2475,"thread_start_us":2214,"threads_started":1}
WARNING: no policy specified for kudu/127.7.39.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.61@KRBTEST.COM" created.
I20260811 12:21:32.600314 11945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:21:32.601765 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.601217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55576)
0811 12:21:32.601408 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.601608 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 12:21:32.602061 11874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
I20260811 12:21:32.603840 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.603350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55584)
0811 12:21:32.603539 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.603653 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 12:21:32.604081 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:33509: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab.
Entry for principal kudu/127.7.39.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab.
I20260811 12:21:32.641628 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.61:33509
--webserver_interface=127.7.39.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab
--principal=kudu/127.7.39.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:38923,127.7.39.61:33509,127.7.39.60:35675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:21:32.653544 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } attempt: 1
I20260811 12:21:32.654989 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.654456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55588)
0811 12:21:32.654691 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.654824 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 12:21:32.656311 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.655806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55596)
0811 12:21:32.656045 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.656140 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260811 12:21:32.656534 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:33509: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
I20260811 12:21:32.724864 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } attempt: 2
I20260811 12:21:32.726162 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.725613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55612)
0811 12:21:32.725882 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.725983 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 12:21:32.727288 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.726871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55618)
0811 12:21:32.727039 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.727131 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:21:32.727492 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:33509: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
I20260811 12:21:32.818766 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } attempt: 3
I20260811 12:21:32.820034 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.819552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55630)
0811 12:21:32.819784 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.819882 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 12:21:32.821075 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.820695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55646)
0811 12:21:32.820846 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.820923 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:21:32.821321 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:33509: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
I20260811 12:21:32.964629 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } attempt: 4
I20260811 12:21:32.966121 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.965504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55658)
0811 12:21:32.965875 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.965972 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
I20260811 12:21:32.967331 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:32.966897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55662)
0811 12:21:32.967071 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:32.967179 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 12:21:32.967562 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.61:33509: Network error: Client connection negotiation failed: client connection to 127.7.39.61:33509: connect: Connection refused (error 111)
W20260811 12:21:33.004235 11954 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:21:33.004647 11954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:33.004752 11954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:33.004871 11954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:33.012517 11954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:33.012655 11954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:33.012749 11954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:33.012817 11954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:33.012905 11954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:33.023307 11954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:38923,127.7.39.61:33509,127.7.39.60:35675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.61.keytab
--principal=kudu/127.7.39.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.61:33509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.39.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:33.024785 11954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:33.026232 11954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:33.039017 11960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:33.039094 11959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:33.039068 11963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:33.039840 11954 server_base.cc:1034] running on GCE node
I20260811 12:21:33.040627 11954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:33.042197 11954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:33.043434 11954 hybrid_clock.cc:648] HybridClock initialized: now 1786450893043361 us; error 48 us; skew 500 ppm
I20260811 12:21:33.043776 11954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:33.050851 11954 init.cc:377] Logged in from keytab as kudu/127.7.39.61@KRBTEST.COM (short username kudu)
I20260811 12:21:33.052691 11954 webserver.cc:460] Webserver started at http://127.7.39.61:45651/ using document root <none> and password file <none>
I20260811 12:21:33.053560 11954 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:33.053687 11954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:33.054041 11954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:33.057466 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/data/instance:
uuid: "bae726d3888b4c4ca7e4a6dd53ed80be"
format_stamp: "Formatted at 2026-08-11 12:21:33 on dist-test-slave-rmhg"
server_key: "a98a17712323c5416b2eb7fc780eac8a"
server_key_iv: "ead8f69589e3815765e877c710a4006b"
server_key_version: "encryptionkey@0"
I20260811 12:21:33.058352 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/wal/instance:
uuid: "bae726d3888b4c4ca7e4a6dd53ed80be"
format_stamp: "Formatted at 2026-08-11 12:21:33 on dist-test-slave-rmhg"
server_key: "a98a17712323c5416b2eb7fc780eac8a"
server_key_iv: "ead8f69589e3815765e877c710a4006b"
server_key_version: "encryptionkey@0"
I20260811 12:21:33.064026 11954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:21:33.067796 11975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:33.069314 11954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:21:33.069504 11954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/wal
uuid: "bae726d3888b4c4ca7e4a6dd53ed80be"
format_stamp: "Formatted at 2026-08-11 12:21:33 on dist-test-slave-rmhg"
server_key: "a98a17712323c5416b2eb7fc780eac8a"
server_key_iv: "ead8f69589e3815765e877c710a4006b"
server_key_version: "encryptionkey@0"
I20260811 12:21:33.069731 11954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:33.093291 11954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:33.098253 11954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:33.098539 11954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:33.117501 11954 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.61:33509
I20260811 12:21:33.117538 12063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.61:33509 every 8 connection(s)
I20260811 12:21:33.118934 11954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/data/info.pb
I20260811 12:21:33.123162 12064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:33.128345 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 11954
I20260811 12:21:33.128561 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-1/wal/instance
I20260811 12:21:33.129091 7324 external_mini_cluster.cc:1428] Setting key 83a03d5b0909ef6b41049dd6522486a0
I20260811 12:21:33.131261 12064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:33.154438 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.134389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.7.39.62:38923)
0811 12:21:33.134893 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.134898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.136351 (+ 1453us) server_negotiation.cc:407] Connection header received
0811 12:21:33.138544 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.138551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.138639 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.138746 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.140578 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.141372 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.143031 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.143480 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.143619 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.147841 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.147846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.147850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.147891 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.150807 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.151710 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.151714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.151717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.151851 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.152312 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.152316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.152318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.152618 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.152779 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.153763 (+ 984us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.154023 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":272,"threads_started":1}
I20260811 12:21:33.155633 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.135171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:38923 (local address 127.0.0.1:46030)
0811 12:21:33.136144 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.136196 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:33.138193 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:33.139045 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:33.139055 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:33.139695 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:33.140328 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.140348 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.141551 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.141556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.142861 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.142876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.143755 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.143763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.144016 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.144823 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:33.144856 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:33.147556 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:33.151006 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.151017 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.151033 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.151562 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.151973 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.151978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.151981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.152186 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.152762 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:33.152771 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:33.153269 (+ 498us) client_negotiation.cc:769] Sending connection context
0811 12:21:33.153663 (+ 394us) client_negotiation.cc:240] Negotiation successful
0811 12:21:33.154294 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":433,"threads_started":1}
I20260811 12:21:33.157925 12064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.39.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.39.60@KRBTEST.COM" created.
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:21:33.174392 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.158946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55664)
0811 12:21:33.159176 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.159195 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:33.159302 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:33.160502 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:33.160506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:33.161001 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:33.161142 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.161216 (+ 74us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.162710 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.162714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.163640 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.163655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.164573 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.164578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.164756 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.165517 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:33.165538 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:33.167873 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:33.171814 (+ 3941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.171820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.171824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.172236 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.172656 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.172661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.172665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.172821 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.173547 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:33.173552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:33.173723 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 12:21:33.173865 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 12:21:33.174122 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 12:21:33.175474 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.159275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55664 (local address 127.7.39.61:33509)
0811 12:21:33.159999 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.160006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.160037 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:21:33.160092 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.160098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.160316 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.160488 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.161379 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.162540 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.163797 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.164390 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.164532 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.168135 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.168153 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.168173 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.168215 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.171605 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.172390 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.172399 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.172401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.172537 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.172929 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.172933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.172935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.173379 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.173606 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.174878 (+ 1272us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.175178 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":403,"threads_started":1}
I20260811 12:21:33.178098 12064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:21:33.179320 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.178839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56316)
0811 12:21:33.179031 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.179173 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 12:21:33.179630 11976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.181275 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.180890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56324)
0811 12:21:33.181037 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.181117 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:21:33.181522 12064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.39.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab.
Entry for principal kudu/127.7.39.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab.
I20260811 12:21:33.209293 7324 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
/tmp/dist-test-taskDQu5eT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.60:35675
--webserver_interface=127.7.39.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab
--principal=kudu/127.7.39.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.39.20:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.39.62:38923,127.7.39.61:33509,127.7.39.60:35675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:21:33.230883 12064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 1
I20260811 12:21:33.232263 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.231727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56336)
0811 12:21:33.231948 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.232063 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 12:21:33.233548 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.233121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56338)
0811 12:21:33.233330 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.233429 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 12:21:33.233764 12064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.266844 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } attempt: 5
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:21:33.282940 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.267898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.7.39.61:33509)
0811 12:21:33.268111 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.268117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.268141 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:21:33.268277 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.268283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.268373 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.268463 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.269568 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.270354 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.272166 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.272675 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.272780 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.277019 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.277026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.277029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.277081 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.279756 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.280712 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.280718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.280720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.280886 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.281407 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.281413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.281415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.281755 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.281887 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.282487 (+ 600us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.282696 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 12:21:33.282986 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.267716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55674)
0811 12:21:33.267955 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.267979 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:33.268131 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:33.268528 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:33.268533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:33.269077 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:33.269384 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.269399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.270533 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.270544 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.271960 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.271992 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.272861 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.272868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.273088 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.273843 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:33.273869 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:33.276744 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:33.279939 (+ 3195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.279945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.279950 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.280543 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.281015 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.281020 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.281024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.281254 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.281886 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:33.281892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:33.282089 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 12:21:33.282251 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 12:21:33.282618 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 12:21:33.284264 11945 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:21:33.285668 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.285200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56342)
0811 12:21:33.285374 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.285462 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:21:33.286881 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.286460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56358)
0811 12:21:33.286650 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.286752 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 12:21:33.287127 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.302026 12064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 2
I20260811 12:21:33.303310 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.302806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56360)
0811 12:21:33.303031 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.303142 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 12:21:33.304495 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.304067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56368)
0811 12:21:33.304230 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.304309 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 12:21:33.304701 12064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.338384 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 1
I20260811 12:21:33.339745 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.339215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56380)
0811 12:21:33.339456 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.339565 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 12:21:33.340775 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.340432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56384)
0811 12:21:33.340593 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.340666 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 12:21:33.341003 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.395958 12064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 3
I20260811 12:21:33.397357 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.396788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56392)
0811 12:21:33.397031 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.397131 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 12:21:33.398597 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.398116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56404)
0811 12:21:33.398351 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.398432 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260811 12:21:33.398811 12064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.409269 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 2
I20260811 12:21:33.410459 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.409943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56410)
0811 12:21:33.410181 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.410276 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 12:21:33.411410 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.411074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56416)
0811 12:21:33.411221 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.411294 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:21:33.411651 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.517863 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 3
I20260811 12:21:33.519100 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.518624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56430)
0811 12:21:33.518826 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.518936 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 12:21:33.520171 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.519765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56434)
0811 12:21:33.519973 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.520061 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 12:21:33.520382 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.542032 12064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 4
I20260811 12:21:33.543210 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.542695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56438)
0811 12:21:33.542934 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.543044 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:21:33.544342 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.543987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56446)
0811 12:21:33.544163 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.544234 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 12:21:33.544593 12064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
W20260811 12:21:33.580430 12074 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:21:33.580821 12074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:21:33.580921 12074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:21:33.581038 12074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:21:33.588543 12074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:21:33.588660 12074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:21:33.588742 12074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:21:33.588811 12074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:21:33.588898 12074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:21:33.598927 12074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.39.20:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.39.62:38923,127.7.39.61:33509,127.7.39.60:35675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/krb5kdc/kudu_127.7.39.60.keytab
--principal=kudu/127.7.39.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.39.60:35675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.39.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:21:33.600306 12074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:21:33.601692 12074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:21:33.614177 12082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:33.614348 12086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:21:33.614382 12081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:21:33.615298 12074 server_base.cc:1034] running on GCE node
I20260811 12:21:33.616185 12074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:21:33.617756 12074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:21:33.618979 12074 hybrid_clock.cc:648] HybridClock initialized: now 1786450893618919 us; error 51 us; skew 500 ppm
I20260811 12:21:33.619333 12074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:21:33.622629 12074 init.cc:377] Logged in from keytab as kudu/127.7.39.60@KRBTEST.COM (short username kudu)
I20260811 12:21:33.624154 12074 webserver.cc:460] Webserver started at http://127.7.39.60:35925/ using document root <none> and password file <none>
I20260811 12:21:33.624905 12074 fs_manager.cc:359] Metadata directory not provided
I20260811 12:21:33.625015 12074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:21:33.625403 12074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:21:33.628584 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/data/instance:
uuid: "e942cbdfcfca435babd0ec51a791243a"
format_stamp: "Formatted at 2026-08-11 12:21:33 on dist-test-slave-rmhg"
server_key: "ed8ed973f090a27cc61be1b45d309d56"
server_key_iv: "7299f942145436ae36246e648a44ba6d"
server_key_version: "encryptionkey@0"
I20260811 12:21:33.629485 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/wal/instance:
uuid: "e942cbdfcfca435babd0ec51a791243a"
format_stamp: "Formatted at 2026-08-11 12:21:33 on dist-test-slave-rmhg"
server_key: "ed8ed973f090a27cc61be1b45d309d56"
server_key_iv: "7299f942145436ae36246e648a44ba6d"
server_key_version: "encryptionkey@0"
I20260811 12:21:33.635339 12074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:21:33.639185 12094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:21:33.640633 12074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:21:33.640831 12074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/data,/tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/wal
uuid: "e942cbdfcfca435babd0ec51a791243a"
format_stamp: "Formatted at 2026-08-11 12:21:33 on dist-test-slave-rmhg"
server_key: "ed8ed973f090a27cc61be1b45d309d56"
server_key_iv: "7299f942145436ae36246e648a44ba6d"
server_key_version: "encryptionkey@0"
I20260811 12:21:33.641047 12074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:21:33.688925 12074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:21:33.693486 12074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:21:33.693781 12074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:21:33.697608 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 4
I20260811 12:21:33.698841 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.698371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56456)
0811 12:21:33.698587 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.698679 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 12:21:33.700189 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.699705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56464)
0811 12:21:33.699893 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.700002 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 12:21:33.700462 11945 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.39.60:35675: Network error: Client connection negotiation failed: client connection to 127.7.39.60:35675: connect: Connection refused (error 111)
I20260811 12:21:33.713076 12074 rpc_server.cc:307] RPC server started. Bound to: 127.7.39.60:35675
I20260811 12:21:33.713099 12170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.39.60:35675 every 8 connection(s)
I20260811 12:21:33.714664 12074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/data/info.pb
I20260811 12:21:33.716544 7324 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu as pid 12074
I20260811 12:21:33.716725 7324 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDQu5eT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450508543501-7324-0/minicluster-data/master-2/wal/instance
I20260811 12:21:33.717216 7324 external_mini_cluster.cc:1428] Setting key c7a4f359daba8856ec31cb9e771ab77c
I20260811 12:21:33.719087 12172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:21:33.728817 12172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
I20260811 12:21:33.742486 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.719064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.7.39.60:35675)
0811 12:21:33.719874 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.719891 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.719944 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:21:33.722095 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.722125 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.722667 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.723134 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.724480 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.726583 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.728112 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.729331 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.729617 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.734541 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.734553 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.734565 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.734611 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.738028 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.738794 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.738802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.738809 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.738996 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.739391 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.739395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.739397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.740013 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.740339 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.740799 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.741420 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":304,"threads_started":1}
I20260811 12:21:33.747821 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.732283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.62:38923 (local address 127.0.0.1:46032)
0811 12:21:33.733128 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.733203 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:33.733374 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:33.734178 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:33.734185 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:33.734913 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:33.735246 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.735260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.736787 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.736791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.738013 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.738028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.738810 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.738816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.738977 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.739563 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:33.739581 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:33.741697 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:33.745048 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.745057 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.745062 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.745603 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.746142 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.746147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.746151 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.746334 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.746987 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:33.746993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:33.747149 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 12:21:33.747294 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 12:21:33.747539 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":399,"threads_started":1}
I20260811 12:21:33.748107 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.732339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.7.39.62:38923)
0811 12:21:33.733806 (+ 1467us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.733811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.733826 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:21:33.733877 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.733881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.733972 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.734070 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.735459 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.736616 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.738193 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.738620 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.738745 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.741898 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.741903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.741907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.741942 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.744915 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.745776 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.745782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.745785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.745984 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.746450 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.746454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.746456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.746847 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.747017 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.747453 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.747690 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":1191,"threads_started":1}
I20260811 12:21:33.749595 12172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.62@KRBTEST.COM
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:21:33.762036 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.745668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46042 (local address 127.7.39.62:38923)
0811 12:21:33.746271 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.746276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.746300 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:21:33.746348 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.746352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.746437 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.746525 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.748120 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.749220 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.750786 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.751390 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.751520 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.756032 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.756040 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.756045 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.756088 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.759080 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.759891 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.759896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.759898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.760037 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.760442 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.760445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.760447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.760739 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.760954 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.761386 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.761619 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":293,"threads_started":1}
I20260811 12:21:33.764209 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.750382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.61:33509 (local address 127.0.0.1:55676)
0811 12:21:33.750551 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.750568 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:33.750646 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:33.750959 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:33.750964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:33.751431 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:33.751673 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.751686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.752959 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.752963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.753913 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.753926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.754771 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.754777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.754969 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.755507 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:33.755525 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:33.757949 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:33.761307 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.761312 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.761316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.761845 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.762294 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.762299 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.762302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.762495 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.763242 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:33.763247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:33.763417 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 12:21:33.763571 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 12:21:33.763864 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 12:21:33.764370 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.750418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.7.39.61:33509)
0811 12:21:33.750606 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.750611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.750630 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:21:33.750756 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.750761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.750843 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.750916 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.751826 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.752784 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.754058 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.754594 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.754691 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.758179 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.758185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.758188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.758223 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.761124 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.761970 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.761974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.761977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.762152 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.762662 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.762667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.762670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.763083 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.763252 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.763673 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.763928 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:21:33.765743 12172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.60@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.39.61@KRBTEST.COM
I20260811 12:21:33.779690 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.766503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56492)
0811 12:21:33.766694 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.766716 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:33.766822 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:33.767117 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:33.767120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:33.767492 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:33.767630 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.767638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.768674 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.768680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.769811 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.769825 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.770577 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.770584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.770772 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.771266 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:33.771284 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:33.773510 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:33.777146 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.777204 (+ 58us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.777207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.777600 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.778052 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.778057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.778061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.778231 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.778761 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:33.778764 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:33.778894 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 12:21:33.779045 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 12:21:33.779391 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:21:33.780015 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.766696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.7.39.60:35675)
0811 12:21:33.766852 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.766858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.766875 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:21:33.766921 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.766925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.766997 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.767099 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.767782 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.768530 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.769963 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.770404 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.770500 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.773690 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.773695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.773698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.773728 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.776973 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.777734 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.777740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.777743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.777910 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.778349 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.778353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.778355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.778639 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.778788 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.779140 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.779614 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":181,"server-negotiator.queue_time_us":52}
I20260811 12:21:33.781046 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.764315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.7.39.61:33509)
0811 12:21:33.764785 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.764796 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.764811 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:21:33.764865 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.764870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.764959 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.765119 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.766921 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.767761 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.769326 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.769783 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.769882 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.774497 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.774505 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.774510 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.774570 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.777812 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.778756 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.778762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.778765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.778927 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.779379 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.779384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.779388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.779763 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.780055 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.780567 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.780759 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":593,"threads_started":1}
I20260811 12:21:33.785820 12172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: Bootstrap starting.
I20260811 12:21:33.786360 7324 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:21:33.790050 12172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:33.791200 12172 log.cc:824] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:33.789683 11909 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46042
I20260811 12:21:33.794430 12172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: No bootstrap required, opened a new log
I20260811 12:21:33.801848 12172 raft_consensus.cc:348] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } }
I20260811 12:21:33.802250 12172 raft_consensus.cc:729] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e942cbdfcfca435babd0ec51a791243a, State: Initialized, Role: FOLLOWER
I20260811 12:21:33.802755 12172 consensus_queue.cc:260] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } }
I20260811 12:21:33.803794 12183 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:33.804200 12183 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:33.805289 12172 sys_catalog.cc:564] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:21:33.816287 12199 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:21:33.816452 12199 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:21:33.813933 12074 master_runner.cc:186] Error getting master registration for 127.7.39.62:38923: OK, instance_id { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" instance_seqno: 1786450892546542 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:21:33.824504 12074 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:21:33.843807 12064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 5
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.61@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
I20260811 12:21:33.859735 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.844666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56506)
0811 12:21:33.845002 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.845024 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:33.845206 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:33.845617 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:33.845623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:33.846197 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:33.846432 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.846447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.847562 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.847569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.849127 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.849187 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.850292 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.850299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.850533 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.851284 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:33.851304 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:33.853618 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:33.856870 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.856876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.856881 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.857399 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.857849 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.857855 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.857859 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.858007 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.858654 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:33.858659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:33.858837 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 12:21:33.858965 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 12:21:33.859337 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260811 12:21:33.859741 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.844689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56506 (local address 127.7.39.60:35675)
0811 12:21:33.844921 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.844928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.845104 (+ 176us) server_negotiation.cc:407] Connection header received
0811 12:21:33.845343 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.845349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.845433 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.845558 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.846612 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.847392 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.849342 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.850099 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.850233 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.853847 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.853853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.853856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.853894 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.856689 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.857553 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.857558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.857561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.857723 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.858135 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.858139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.858141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.858518 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.858702 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.859120 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.859361 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:21:33.866241 12064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: Bootstrap starting.
I20260811 12:21:33.871126 12064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:33.872598 12064 log.cc:824] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:33.876283 12064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: No bootstrap required, opened a new log
I20260811 12:21:33.887643 12064 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } }
I20260811 12:21:33.888132 12064 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae726d3888b4c4ca7e4a6dd53ed80be, State: Initialized, Role: FOLLOWER
I20260811 12:21:33.888721 12064 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } }
I20260811 12:21:33.889878 12204 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:33.890341 12204 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:33.891366 12064 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:21:33.902462 12219 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:21:33.902776 12219 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:21:33.899019 11954 master_runner.cc:186] Error getting master registration for 127.7.39.62:38923: OK, instance_id { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" instance_seqno: 1786450892546542 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:21:33.911294 11954 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 12:21:33.946087 12102 tablet.cc:2369] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:21:33.977730 11945 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } attempt: 5
Aug 11 12:21:33 dist-test-slave-rmhg krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.39.62@KRBTEST.COM for kudu/127.7.39.60@KRBTEST.COM
I20260811 12:21:33.997074 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.978521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.39.60:35675 (local address 127.0.0.1:56516)
0811 12:21:33.978984 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 12:21:33.979005 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:21:33.979139 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:21:33.980057 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:21:33.980062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:21:33.980607 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:21:33.980847 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.980859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.982471 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.982477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.984027 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:21:33.984045 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.985417 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.985424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:21:33.985648 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.986368 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:21:33.986394 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:21:33.989429 (+ 3035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:21:33.993954 (+ 4525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.993959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.993964 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.994569 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.995053 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:21:33.995058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:21:33.995061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:21:33.995278 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:21:33.996010 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:21:33.996016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:21:33.996181 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 12:21:33.996327 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 12:21:33.996635 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314}
I20260811 12:21:33.998046 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:33.979365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.7.39.60:35675)
0811 12:21:33.979566 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:33.979572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:33.979605 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:21:33.979670 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:33.979675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:33.979753 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:33.979855 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:33.981058 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.982281 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.984243 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:33.984993 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:33.985120 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:33.989700 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:33.989709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:33.989714 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:33.989764 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:33.993732 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.994737 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.994744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.994748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.994912 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:33.995415 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:33.995421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:33.995424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:33.995799 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:33.995946 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:33.997429 (+ 1483us) server_negotiation.cc:299] Negotiation successful
0811 12:21:33.997692 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:21:34.004626 11945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7: Bootstrap starting.
I20260811 12:21:34.009522 11945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7: Neither blocks nor log segments found. Creating new log.
I20260811 12:21:34.010996 11945 log.cc:824] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7: Log is configured to *not* fsync() on all Append() calls
I20260811 12:21:34.014803 11945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7: No bootstrap required, opened a new log
I20260811 12:21:34.022725 11945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } }
I20260811 12:21:34.023221 11945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1d7fc8c67e406d946d7c35b6fe2ea7, State: Initialized, Role: FOLLOWER
I20260811 12:21:34.023782 11945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } }
I20260811 12:21:34.025127 12223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:34.025945 12223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:34.026310 11945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:21:34.026371 11909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 12:21:34.026582 11909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:34.030076 11909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } }
I20260811 12:21:34.032603 12024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be"
I20260811 12:21:34.033195 12024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:21:34.033468 12136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e942cbdfcfca435babd0ec51a791243a"
I20260811 12:21:34.033969 12136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:21:34.035789 11876 tablet.cc:2369] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:21:34.036623 12024 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e1d7fc8c67e406d946d7c35b6fe2ea7 in term 1.
I20260811 12:21:34.037792 11874 leader_election.cc:302] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1d7fc8c67e406d946d7c35b6fe2ea7, bae726d3888b4c4ca7e4a6dd53ed80be; no voters:
I20260811 12:21:34.038801 12136 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e1d7fc8c67e406d946d7c35b6fe2ea7 in term 1.
I20260811 12:21:34.039572 11909 leader_election.cc:288] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [CANDIDATE]: Term 1 election: Requested vote from peers bae726d3888b4c4ca7e4a6dd53ed80be (127.7.39.61:33509), e942cbdfcfca435babd0ec51a791243a (127.7.39.60:35675)
I20260811 12:21:34.046720 12223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:21:34.053495 12223 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [term 1 LEADER]: Becoming Leader. State: Replica: 4e1d7fc8c67e406d946d7c35b6fe2ea7, State: Running, Role: LEADER
I20260811 12:21:34.054592 12223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } }
I20260811 12:21:34.066217 12231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e1d7fc8c67e406d946d7c35b6fe2ea7. Latest consensus state: current_term: 1 leader_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:34.066902 12231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:34.071131 12241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:21:34.090132 12241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 12:21:34.101438 11984 tablet.cc:2369] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:21:34.108876 12260 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:21:34.131110 12136 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [term 1 FOLLOWER]: Refusing update from remote peer 4e1d7fc8c67e406d946d7c35b6fe2ea7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:21:34.132571 12231 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:21:34.144397 12024 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [term 1 FOLLOWER]: Refusing update from remote peer 4e1d7fc8c67e406d946d7c35b6fe2ea7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:21:34.145691 12231 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:21:34.146834 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:34.081974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56526 (local address 127.7.39.60:35675)
0811 12:21:34.082187 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.082193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.098954 (+ 16761us) server_negotiation.cc:407] Connection header received
0811 12:21:34.101927 (+ 2973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.101934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.102022 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.102100 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.108025 (+ 5925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:34.109232 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:34.117586 (+ 8354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:34.118305 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:34.118409 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:34.132530 (+ 14121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:34.132537 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:34.132542 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:34.132606 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:34.136507 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:34.140335 (+ 3828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:34.140343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:34.140347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:34.140550 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:34.141831 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:34.141838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:34.141842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:34.142272 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:34.142553 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:34.146108 (+ 3555us) server_negotiation.cc:299] Negotiation successful
0811 12:21:34.146419 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:21:34.149205 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:34.079986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.7.39.62:38923)
0811 12:21:34.080214 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.080221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.097965 (+ 17744us) server_negotiation.cc:407] Connection header received
0811 12:21:34.099254 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.099261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.099380 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.099465 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.104493 (+ 5028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:34.105755 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:34.115466 (+ 9711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:34.116147 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:34.116248 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:34.127541 (+ 11293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:34.127549 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:34.127555 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:34.127623 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:34.132239 (+ 4616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:34.141085 (+ 8846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:34.141092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:34.141095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:34.141442 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:34.144604 (+ 3162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:34.144610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:34.144617 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:34.145045 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:34.145394 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:34.148533 (+ 3139us) server_negotiation.cc:299] Negotiation successful
0811 12:21:34.148807 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:21:34.161233 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:34.095999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55700 (local address 127.7.39.61:33509)
0811 12:21:34.096203 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.096209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.112131 (+ 15922us) server_negotiation.cc:407] Connection header received
0811 12:21:34.112658 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.112664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.112756 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.112836 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.124091 (+ 11255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:34.125339 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:34.138354 (+ 13015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:34.139090 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:34.139199 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:34.153284 (+ 14085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:21:34.153294 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:21:34.153299 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:21:34.153353 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:21:34.157371 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:34.158497 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:34.158504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:34.158508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:34.158708 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:21:34.159206 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:21:34.159212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:21:34.159215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:21:34.159673 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:21:34.159927 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:34.160530 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 12:21:34.160816 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:21:34.165622 12204 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e1d7fc8c67e406d946d7c35b6fe2ea7. Latest consensus state: current_term: 1 leader_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:34.165993 12204 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:34.173622 12204 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:34.173955 12204 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:34.184680 12264 mvcc.cc:205] Tried to move back new op lower bound from 7317302862347657216 to 7317302862064910336. Current Snapshot: MvccSnapshot[applied={T|T < 7317302862347657216}]
I20260811 12:21:34.191133 12183 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e1d7fc8c67e406d946d7c35b6fe2ea7. Latest consensus state: current_term: 1 leader_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:34.191473 12183 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:34.192535 12223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:34.192912 12223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:34.193295 12223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:34.193529 12223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:21:34.194329 12183 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1d7fc8c67e406d946d7c35b6fe2ea7" member_type: VOTER last_known_addr { host: "127.7.39.62" port: 38923 } } peers { permanent_uuid: "bae726d3888b4c4ca7e4a6dd53ed80be" member_type: VOTER last_known_addr { host: "127.7.39.61" port: 33509 } } peers { permanent_uuid: "e942cbdfcfca435babd0ec51a791243a" member_type: VOTER last_known_addr { host: "127.7.39.60" port: 35675 } } }
I20260811 12:21:34.194599 12183 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:21:34.195779 12241 catalog_manager.cc:1347] Generated new cluster ID: 7925a9bd30c44392b44bc922ba82a307
I20260811 12:21:34.195890 12241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:21:34.213771 12241 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:21:34.218674 12241 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:21:34.238790 12241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e1d7fc8c67e406d946d7c35b6fe2ea7: Generated new TSK 0
I20260811 12:21:34.240175 12241 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:21:34.399804 7324 mini_kdc.cc:348] Time spent kdestroy: real 0.131s user 0.002s sys 0.031s
W20260811 12:21:34.432130 12178 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.414349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.7.39.62:38923)
0811 12:21:34.414562 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.414569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.429805 (+ 15236us) server_negotiation.cc:407] Connection header received
0811 12:21:34.430141 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.430148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.430254 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.430348 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.431768 (+ 1420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46048: BlockingRecv error: recv got EOF from 127.0.0.1:46048 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 12:21:34.434839 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.424761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55702 (local address 127.7.39.61:33509)
0811 12:21:34.424981 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.424988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.432740 (+ 7752us) server_negotiation.cc:407] Connection header received
0811 12:21:34.432990 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.432996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.433079 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.433145 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.434590 (+ 1445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55702: BlockingRecv error: recv got EOF from 127.0.0.1:55702 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 12:21:34.437054 12175 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.420978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.7.39.60:35675)
0811 12:21:34.421230 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.421237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.431226 (+ 9989us) server_negotiation.cc:407] Connection header received
0811 12:21:34.432918 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.432925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.433038 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.433114 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.436756 (+ 3642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56540: BlockingRecv error: recv got EOF from 127.0.0.1:56540 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260811 12:21:34.488967 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.471191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55704 (local address 127.7.39.61:33509)
0811 12:21:34.471430 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.471436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.486614 (+ 15178us) server_negotiation.cc:407] Connection header received
0811 12:21:34.486945 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.486953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.487147 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.487268 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.488638 (+ 1370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55704: BlockingRecv error: recv got EOF from 127.0.0.1:55704 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 12:21:34.530774 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.521303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.7.39.61:33509)
0811 12:21:34.521513 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.521518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.528610 (+ 7092us) server_negotiation.cc:407] Connection header received
0811 12:21:34.529221 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.529229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.529336 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.529413 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.530533 (+ 1120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55720: BlockingRecv error: recv got EOF from 127.0.0.1:55720 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 12:21:34.579767 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.570161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.7.39.61:33509)
0811 12:21:34.570366 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.570373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.577636 (+ 7263us) server_negotiation.cc:407] Connection header received
0811 12:21:34.578199 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.578206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.578327 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.578403 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.579509 (+ 1106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55728: BlockingRecv error: recv got EOF from 127.0.0.1:55728 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 12:21:34.591409 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.588265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.7.39.61:33509)
0811 12:21:34.588504 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.588510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.589801 (+ 1291us) server_negotiation.cc:407] Connection header received
0811 12:21:34.590025 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.590031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.590136 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.590261 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.591156 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55736: BlockingRecv error: recv got EOF from 127.0.0.1:55736 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
W20260811 12:21:34.608381 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.605671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.7.39.61:33509)
0811 12:21:34.605932 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.605939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.606601 (+ 662us) server_negotiation.cc:407] Connection header received
0811 12:21:34.606855 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.606862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.606981 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.607113 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.608132 (+ 1019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55750: BlockingRecv error: recv got EOF from 127.0.0.1:55750 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 12:21:34.630918 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.628418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55752 (local address 127.7.39.61:33509)
0811 12:21:34.628613 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.628620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.629229 (+ 609us) server_negotiation.cc:407] Connection header received
0811 12:21:34.629474 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.629481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.629598 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.629716 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.630671 (+ 955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55752: BlockingRecv error: recv got EOF from 127.0.0.1:55752 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 12:21:34.669605 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.666872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.7.39.61:33509)
0811 12:21:34.667069 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.667075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.667686 (+ 611us) server_negotiation.cc:407] Connection header received
0811 12:21:34.667926 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.667933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.668059 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.668189 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.669205 (+ 1016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55758: BlockingRecv error: recv got EOF from 127.0.0.1:55758 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 12:21:34.739965 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.738006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.7.39.61:33509)
0811 12:21:34.738241 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.738249 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.738512 (+ 263us) server_negotiation.cc:407] Connection header received
0811 12:21:34.738782 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.738787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.738865 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.738957 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.739775 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55772: BlockingRecv error: recv got EOF from 127.0.0.1:55772 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 12:21:34.818758 12199 catalog_manager.cc:1259] Loaded cluster ID: 7925a9bd30c44392b44bc922ba82a307
I20260811 12:21:34.818879 12199 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: loading cluster ID for follower catalog manager: success
I20260811 12:21:34.821696 12199 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: acquiring CA information for follower catalog manager: success
I20260811 12:21:34.824312 12199 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e942cbdfcfca435babd0ec51a791243a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 12:21:34.873726 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:21:34.871602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.7.39.61:33509)
0811 12:21:34.871827 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:34.871834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:34.872193 (+ 359us) server_negotiation.cc:407] Connection header received
0811 12:21:34.872430 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:34.872434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:34.872511 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:34.872604 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:21:34.873514 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55776: BlockingRecv error: recv got EOF from 127.0.0.1:55776 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:21:34.905119 12219 catalog_manager.cc:1259] Loaded cluster ID: 7925a9bd30c44392b44bc922ba82a307
I20260811 12:21:34.905283 12219 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: loading cluster ID for follower catalog manager: success
I20260811 12:21:34.908022 12219 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: acquiring CA information for follower catalog manager: success
I20260811 12:21:34.910090 12219 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bae726d3888b4c4ca7e4a6dd53ed80be: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:21:35.141288 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:35.135197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55780 (local address 127.7.39.61:33509)
0811 12:21:35.135378 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:35.135383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:35.135632 (+ 249us) server_negotiation.cc:407] Connection header received
0811 12:21:35.135802 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:35.135806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:35.135981 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:35.136098 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:35.136583 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.137344 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.138848 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.139257 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.139357 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:35.139979 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:35.140614 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:35.140733 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:35.140917 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 12:21:35.141003 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:21:35.153744 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:35.147732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56552 (local address 127.7.39.60:35675)
0811 12:21:35.148205 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:35.148210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:35.148229 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:21:35.148431 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:35.148435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:35.148592 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:35.148774 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:35.149205 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.149977 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.151347 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.151747 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.151846 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:35.152395 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:35.152979 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:35.153113 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:35.153329 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 12:21:35.153408 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":244,"threads_started":1}
I20260811 12:21:35.176623 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:35.160217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.7.39.60:35675)
0811 12:21:35.160602 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:35.160609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:35.170356 (+ 9747us) server_negotiation.cc:407] Connection header received
0811 12:21:35.170665 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:35.170671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:35.170759 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:35.170823 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:35.171603 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.172476 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.174351 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.175047 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.175196 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:35.175652 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:35.175783 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:35.175873 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:35.176019 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:21:35.176111 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260811 12:21:35.176610 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:35.160004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55794 (local address 127.7.39.61:33509)
0811 12:21:35.160424 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:35.160431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:35.170015 (+ 9584us) server_negotiation.cc:407] Connection header received
0811 12:21:35.170362 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:35.170367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:35.170447 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:35.170534 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:35.171070 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.171950 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.174417 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.175114 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.175228 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:35.175812 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:35.175947 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:35.176040 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:35.176237 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 12:21:35.176322 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260811 12:21:35.177085 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:21:35.159847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.7.39.62:38923)
0811 12:21:35.169212 (+ 9365us) server_negotiation.cc:206] Beginning negotiation
0811 12:21:35.169221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:21:35.169248 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:21:35.169407 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:21:35.169412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:21:35.169671 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:21:35.170652 (+ 981us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:21:35.170853 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.171889 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.174401 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:21:35.174866 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:21:35.174995 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:21:35.175623 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:21:35.176130 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:21:35.176631 (+ 501us) server_negotiation.cc:1035] Waiting for connection context
0811 12:21:35.176683 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 12:21:35.176777 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9179,"thread_start_us":9030,"threads_started":1}
I20260811 12:21:35.181941 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11833
I20260811 12:21:35.183194 11939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:35.295475 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11833
I20260811 12:21:35.325199 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 11954
I20260811 12:21:35.326440 12056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:35.438242 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 11954
I20260811 12:21:35.468636 7324 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu pid 12074
I20260811 12:21:35.469919 12163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:21:35.579818 7324 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDQu5eT/build/asan/bin/kudu with pid 12074
2026-08-11T12:21:35Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5152 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10167 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (386959 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST